Buteo/SyncML

Introduction

This wiki document describes the Buteo SyncML stack class design. The SyncML stack was written in C++ and has a Qt like API. This document can be used by any application developer who wants to use the SyncML stack to create SyncML based application. For a complete class description and design, refer to the doxygen documentation

Code: http://meego.gitorious.org/meego-middleware/buteo-syncml/trees/master

Class Design

Following is the context diagram of SyncML stack w.r.t the dependencies it has on other components

Here is a list of external libraries SyncML Stack depends on.

• libqt4-dev (>= 4.5) - for Qt Core , Qt Network , Qt Xml Interfaces.
• xml2wbxml-dev - for xml2wbxml conversion
• libsqlite3-dev - for sqlite3 queries in dealing with changelog and other dbs
• libopenobex1-dev – for Obex Communication
• libqt4-sql-sqlite - Need this library for running and loading sqlite3 drivers in arm environment to run unit tests. This is not a direct dependency for compiling the code.
• sqlite3 – Core sqlite3 Library for sqlite3 commands. There is no compilation dependency , but is needed for running unit tests in scratchbox arm environment.

Functional Description

SyncML Stack has the following Important Components/Classes in the System. External Interfaces used/set by the user of the stack.

• SyncAgent
• SyncAgentConfig
• HTTPTransport
• OBEXTransport

Callback Interface Implemented by the user of the Stack

• StoragePlugin

Internal Interfaces ( used internally in the stack)

• SessionHandler
• StorageHandler
• CommandHandler
• SyncMLMessageParser
• ResponseGenerator

The overall class diagram for the stack in the framework is depicted as shown below.

The following Sections describe the functionality and interaction between each of these components in the overall system.

SyncAgent

SyncAgent acts as an external interface to the Stack and is a single point of entry into the Stack. Anyone using the SyncAgent should create it in the run function of a thread, as SyncAgent further creates all the objects necessary as children of SyncAgent. Qt expects both the parent and child of a object in one thread of execution for Signal/slot Mechanism to work.

SyncAgent has the following important API exposed to the components outside the stack. The API is same for both client and server modes of the stack. PrepareSync StartSync PauseSync ResumeSync AbortSync GetLastError GetLastSyncResults GetLastSyncState

PrepareSync takes SyncAgentConfig Class as the input parameter and gives the configuration details to all the components of the system. PrepareSync is used to prepare the stack to handle further sync requests and as such this is a pre-requisite call for startSync in both client and server modes. PrepareSync returns a bool to indicate if the command was initiated and user should expect a callback when command has been completed.

StartSync is used to trigger the synchronization in case of client mode and in case of server mode this function triggers the transport to send data to stack for parsing. StartSync triggers the need to send/handle the SyncML requests by the stack.

PauseSync, resumeSync, abortSync are the APIs to control the synchronization process and such do their designated jobs. GetLastError, GetLastSyncResults, GetLastSyncStatus give the user information about the status and results of the last sync done in the stack. All the APIs of the SyncAgent are reentrant and but not thread safe.

SyncAgentConfig

SyncAgentConfig is the class used to configure the sync details like sync url, sync account username, password, local and remote database uri’s, setting the transport ( http/obex ),setting the conflict resolution policy details ,setting operation mode of the stack ( client /server ) , enable or disable the stack logging etc., All other components in the stack use SyncAgentConfig to retrieve information about the present sync process using this object.

It has the following functions to set configuration details related to Sync setTransport – sets the transport object to be used by the stack setSyncMode – sets the SyncMode object for stack( slow-sync, two-way–sync ,one-way-sync) setLocalDevice - set local database uri information setRemoteDevice - set the remote database uri information setAuthenticationType – set type of authentication ( no authentication, basic, md5 ) setUsername – set the sync account username to be used for sync setPassword – set the sync account password to be used for sync setDatabaseFilePath – set the db file path for storing persistent information setConflictResolutionPolicy – set the polcify to use in case of conflicts during sync. setOperationMode – set the client/server operation mode of stack addSource - used to add more source databases ( like calendar etc.,) to sync. setProtocolVersion – sets protocol version to operate on. Valid values are DS_1_2, DS_1_1 of SyncML as mentioned in the data structure ProtocolVersion. setProtocolExtensions – used to set ProtocolExtensions needed as in the case of Sync with Series 60 phones.

In addition to these there are certain utility functions used to configure the logging capabilities of the stack. These are enableLogging disableLogging enableLogLevel disableLogLevel

The various loglevels that can be enabled are LogLevel_Debug, LogLevel_Warning, LogLevel_Critical, LogLevel_Fatal, LogLevel_Info, LogLevel_Trace, LogLevel_Protocol

The user of the stack has two options to set the configuration of the stack

• One way is to use the SyncAgentConfig object directly and set the various properties
• The second way is to use the stack configuration file to set the default values for the stack object. The default values in the XML file can be overridden by using the SyncAgentConfig object. A sample XML file is as under

 <?xml version="1.0" encoding="UTF-8"?>
 <meego-syncml-conf xmlns="http://meego.org/meego-syncml-conf">
   <dbpath>syncml.db</dbpath>
   <local-device-name>MeegoDevice</local-device-name>
   <agent-props>
       <max-message-size>16384</max-message-size>
       <max-changes-per-message>22</max-changes-per-message>
       <conflict-resolution-policy>0</conflict-resolution-policy>
   </agent-props>
   <transport-props>
       <bt-obex-mtu>1024</bt-obex-mtu>
       <usb-obex-mtu>1024</usb-obex-mtu>
       <http-number-of-resend-attempts>3</http-number-of-resend-attempts>
   </transport-props>
 </meego-syncml-conf>

HTTPTransport

This is the transport Object that the user of the stack has to create and set in SyncAgentConfig. Stack uses this object to communicate to the remote end using http/https protocol as specified by the RemoteLocUri of this object.

Important public API in this object are setRemoteLocURI – use to set the remote sync uri setProxyConfig – use to set the proxy configuration setWbXml – use to set the content type to WbXML

OBEXTransport

This object needs to be set in case of communication over OBEX is preferred. OBEX connection can be established using BT address and service UID, or in the case of existing connection, using a file descriptor. Currently only OBEX over BT is supported. OBEX over USB is planned but not implemented at this time.

StoragePlugin

StoragePlugin is a callback interface to the storageplugins of the framework for adding, removing a SyncItem (like an email or bookmark), and read or write data to it. The actual writing of data to an item is done via the SyncItem class methods. Important Public API provided by the StoragePlugin class are : generateSyncItem – used to generate an empty Sync Item getSyncItem – used to get the SyncItem from the StoragePlugin commitSyncItem – called after generateSyncItem to write the actual data removeSyncItem – used to delete the SyncItem from the StoragePlugin getAll – used to get all SyncItems from the StoragePlugin getNew – used to get SyncItems created after specified timestamp getModified – used to get SyncItems modified after specified timestamps

SessionHandler

Session Handler is the most important class in the entire stack that co-ordinates activities between different components,delegates tasks to other components, maintains the state machine for the stack in both client and server mode. It interacts with transport to get and send data over the transport. It connects Transport and SyncMLMessageParser to handle incoming data. It co-ordinates between Transport and ResponseGenerator to send outgoing data. Whenever it receives signals from the Parser, it delegates the task of handling the SyncML Commands that are not related to SyncML session to Command Handler.

The high level logic of sending data to remote location has been divided between Session Handler and Response Generator. Based on current state of the synchronization session, Session Handler determines what needs to be sent to server. It uses Packages to form the response needed for a request made from remote side. Packages are logical entities that comprise of data to be sent. A Package is a distinct part of SyncML message that handles dividing data into multiple messages by tracking the size of data inserted into SyncML message before being sent. Package tracks the state of the data sent and splits itself into multiple messages if the size of the message is more than that is supported by the remote device. Packages generate SyncML XML documents by utilizing SyncMLCmdObjects. SyncMLCmdObjects are able to transform their data contents to XML. SyncMLCmdObjects are also able to form hierarchies and such represent the base objects needed for SyncML protocol information exchange.

SessionHandler in its hierarchy has two forms depending on the present Operating Mode. If the Operating Mode is client, specific jobs needed in this mode are handled by Client Session Handler. For example, initiating the sync process, forming init message and sending it across. If the Operating Mode is server, specific jobs needed in this mode are handled by Server Session Handler. For example, Server specific functionality like server state information, alert command handling etc., is taken care by this Class. Base class Session Handler takes care of all the functionality common between client and server like, reading the response of the requests made and delegating the task of understanding commands to command handler.

SessionHandler has the following important API.

• prepareSync - this is a function in that is common for both server and client. It handles preparation of synchronization session.
• startSync - this is a pure virtual function in baseclass and has its own functionality in client and server modes. In Client Mode , this function initiates the Sync for client intiated sync and in server initiated sync waits for the data to be read from the transport. In Server Mode , this function initiates the sync request for server initiated sync and waits for the data from transport in case of client initiated sync.
• sendNextMessage - this API uses Response Generator to send the next Message based on the packages available in the current package Queue. Sending functionality is same for both client and server modes.
• composeLocalChangesPackage - populates package queue with packages that together compose the local changes
• saveSession – Saves the complete Session Information , like Changes to Changelog file , Mapping Table , Anchors etc.,
• validateConfiguration – Validates the SyncAgentConfig Object for Valid Values.
• finishSync – Final step in the sync process. Called when we have no more activities to do in the sync. This function emits a signal to indicate that all the data from the corresponding message for Sync has been read from the transport and the resources specific to the transport for that message can be freed.

Other Tasks related to Session Handler are implemented using Signal/Slot Mechanism of Qt. For Complete List of Functions in Session Handler Please refer to API documentation

StorageHandler

Storage Handler takes care of retrieving/adding/deleting/modifying items in storages. This class provides the API like interface for several storageplugins supported. StorageHandler has the following important API used by the SessionHandler itemBegin – to begin building an item itemAddData – to add data to item being built itemReject – to reject the item that was built itemFinalize – to commit the item was built

CommandHandler

Command Handler Class takes care of handling several SyncML Commands like addItem , delete Item , Replace Item , Put , Get etc., Situations specific to client and server are handled in respective derived classes. Important API in command handler: handleSync – handles the SyncML command Sync handlePut – handles the SyncML command Put handleGet – handles the SyncML command Get handleStatus – handles the SyncML command Status handleMap – handles the SyncML command Map For Information on Other API please refer to the API documentation

ResponseGenerator

ResponseGenerator generates the actual syncml message to be sent to the remote side. Actual sending of requests is done by SessionHandler. ResponseGenerator uses the Packages to Generate the Messages needed to be sent to the other end. Important API in Response Generator: generateNextMessage – Generates a new SyncMLMessage based on package queue addPackage – adds a package to the package queue addStatus – adds a Status to the package queue clearPackageQueue – clears the package queue

SyncMLMessageParser

SyncMLMessageParser is responsible for processing responses received from remote side. It reads data in the slot connected to Transport and parses the incoming xml to several data structures and emits signals with these structures filled with necessary information. This Class Inherits from QXmlStreamReader and adds parsing functionality specific to SyncML. Important Commands of the SyncML Specification that are relevant to Sync’ing with 1.1. and 1.2 protocol support are parsed and the data structures corresponding to individual elements are populated. Important API : parseResponse – this is the slot connected to readData signal of Transport by SessionHandler. There are several other parsing API that parse the header , body , and further elements in the XML formatted message based on the current tokenType() pointed by the QXmlStreamReader. These are private to the class and more information about them can be obtained from API documentation.

Other Classes

LocalChangesPackage - Derived Class of Package Class that takes care of generating the Package Containing Local Changes to be sent to the remote side for Sync. LocalMappingsPackage – Derived Class of Package Class that takes care of Sending the Mappings to Server after receiving items from the remote side. AlertPackage – Derived Class of Package Class that has Alert Parameters set for sending a SyncML Message Containing Alert Command. AuthenticationPackage - Derived Class of Package Class that fills authentication information to SyncML Message header SyncItem – Used by StoragePlugin to describe One SyncItem. A SyncItem can be a contact entry or a calendar entry depending upon the StoragePlugin SyncItemBuilder – Fills up the Data for the SyncItem , based on the data received from the remote side. SycnElements folder – several Classes in syncelements folder that made up individual commands like Add , Alert , Body , Cred , Delete , DevInf ,SyncHdr ,Map , MapItem, Item , Meta,Put , Replace , Results , Status , Sync , and Messages Containing Multiple Commands like , LocalChanges , LocalChange, SyncMLMessage.

Sequence Diagrams

Receiving Sync Command for adding an item from Remote End

Server Flow - Handling First Request

Server Flow for Client Changes without mapping

Server Flow for Client Changes flow with mapping

Session Creation

Sending Items

State Machine

The below diagram depicts the client state machine inside the syncml stack. The several terms used in the state machine are explained in the table below.

Client State Machine

Server State Machine

Sync States

State	Description
UNPREPARED	When a new instance of SyncAgent is created, it is in UNPREPARED state. This means that it is not yet ready for operations as it has not been prepared by passing a proper SyncAgentConfig object to it using prepareSync() method.
PREPARED	When a proper SyncAgentConfig object has been passed to SyncAgent, SyncAgent moves to prepared state. At this point, SyncAgent is ready perform start and resume operations.
LOCAL_INIT	When SyncAgent is in LOCAL_INIT state, it is performing SyncML initialization phase. This phase includes authentication, exchange of device information sync mode negotiations.
REMOTE_INIT	When SyncAgent is in LOCAL_INIT state, it is receiving handshaking information from remote side about SyncML initialization phase
SENDING_ITEMS	When SyncAgent is in SENDING_ITEMS state, it is sending detected changes in local databases to remote side.
RECEIVING_ITEMS	When SyncAgent is in RECEIVING_ITEMS state, it is receiving changes detected by remote side.
SENDING_MAPPINGS	When SyncAgent is in SENDING_MAPPINGS state, it is sending server the local UID mappings of the changes received . Note that when sync direction is client-only, this state never occurs as mappings never need to be sent. In this case, SyncAgent state moves directly from RECEIVING_ITEMS to FINALIZING
RECEIVING_MAPPINGS	When SyncAgent is in RECEIVING_MAPPINGS state, it is receiving the remote UID mappings of the changes sent to remote side.
FINALIZING	When SyncAgent is in FINALIZING state, it is waiting for map acknowledgement package from remote side. After this package has been received, SyncAgent will finalize sync session by storing sync anchors.
SYNC_FINISHED	When SyncAgent is in SYNC_FINISHED state, initiated sync session has completed successfully.

Sync Error States

Error	Description
INVALID_CONFIGURATION	SyncAgentConfig object passed SyncAgent is invalid, for example it might not have proper TransportConnectionData object or it has no sync targets specified.
INTERNAL_ERROR	Internal error has occurred during processing of operation.
ABORTED	Sync session was aborted by user or due to fatal error received in protocol response.
SUSPENDING	User has called suspendSync() method and SyncAgent is currenly negotiating with server about suspending the session.
SUSPENDED	SyncAgent has finished suspending the session.
CONNECTION_ERROR	Transport layer has lost connection to remote side during session.
INVALID_SYNCML_MESSAGE	Received response could not be parsed as the response was empty or contained invalid or unrecognized data.