PIM Storage Requirements

This page explains what PIM data storage systems should provide to synchronize well. Not all of the different aspects are essential, therefore the table includes a column on the impact of not having it. The Evolution Data Server (EDS) and plain files as accessed by the file backend are used as examples.

Feature Description Impact if missing EDS Files
Item import/export in standard format Importing/exporting as vCard/vCalendar/iCalendar string. SyncEvolution supports backends which copy between the internal field list and a PIM storage specific format, but this more work to implement and less flexible because the mapping has to be implemented in C++, compared to a mapping defined in XML when standard formats are supported. Other applications besides SyncEvolution also need text import/export (importing .vcf, .ics, meeting invitations). Supported. Supported; each file contains one item in the standard format.
iCalendar 2.0 Full support for iCalendar 2.0, including all of its features (time zone definitions, meetings, globally unique ID, detached recurrences = VEVENT with UID + RECURRENCE-ID), is expected nowadays. Any less capable implementation will have problems passing the http://www.calconnect.org/publications/MobileCalendarTestSuitev1.1.pdf interoperability tests. When acting as SyncML server, SyncEvolution must support all properties of its peers. Some of these will support iCalendar 2.0 (like Outlook or Evolution). libecal is based on iCalendar 2.0. They are stored literally by the file backend. The file backend supports all formats supported by SyncEvolution itself, which includes iCalendar 2.0.
iCalendar 2.0 VTIMEZONE When receiving meeting invitations, it is often impossible to identify the matching internal timezone, so the original VTIMEZONE has to be stored and used to calculate times. VTIMEZONEs must be attached to the VEVENT/VTODO. Using the wrong timezone rules leads to incorrect times of events and tasks. Storing VTIMEZONE separately is possible, but must ensure that the correct VTIMEZONE for each VEVENT can be looked up. The TZID property is not enough for this, because rules associated with the same TZID have changed over time. libecal has methods to store VTIMEZONEs per calendar. The caller must ensure that the TZID is unique, for which an utility function exits. Each file stores a VCALENDAR, which includes VEVENT/VTODO and matching VTIMEZONE.
iCalendar 2.0 UID + RECURRENCE-ID Different VEVENTs with the same UID belong to the same repeating event. The parent event has a recurrence rule, the child event RECURRENCE-ID which identifies which recurrence they overwrite. The API for importing/exporting items should be able to work on individual VEVENT/VTODO items, not the complete series. All known servers with iCalendar 2.0 support treat one VEVENT as one item. The backend implementation becomes harder when the storage only supports importing/exporting a complete series. It is possible to address VEVENTs with UID+RECURRENCE-ID, but operations without RECURRENCE-ID always affect the whole series. The SyncEvolution backend of EDS needs many special cases to handle all possible combinations of operations. Each file is independent from all others, so this works.
vCard 3.0 plus extensions vCard 3.0 adds several properties not found in vCard 2.1. Several extensions (spouse, anniversary, ...) are in common use. Same impact as for iCalendar 2.0. As for iCalendar 2.0. A minor difference is that the EDS file backend stores vCards in a Berkley DB. See above.
Local ID Items are identified in SyncML by an opaque string which is created locally and must be unique locally. Must not change as the item is changed or the system restarted. When creating an item, it must be possible to retrieve the ID of that item reliably. Without such a local ID, syncing must identify items based on their content, which is error prone. The only way to do this in SyncEvolution is to force a slow sync in each sync session. IDs are part of the libebook and libecal APIs. The libebook ID is used in the backend directly. With libecal, a combination of UID and RECURRENCE-ID is used to identify an item (multiple VEVENTs may have the same UID). The file backend manages files inside a single directory. The filename is used as ID.
Revision String An opaque string for each item which changes if and only if the corresponding item is modified. The TrackingSyncSource uses the ID/Rev String pairs to identify added, deleted and modified items. It keeps a table of the known last state and compares it against the current state to do this. This works for multiple peers by making that remembered state peer-specific. Without a revision string, a backend must implement its own change tracking. LAST-MODIFIED (calendar) and REV (contacts) are used. Most EDS backends update those, as expected by SyncEvolution. CouchDB did not originally support REV, but was enhanced in response to a bug report about it. The last modified time stamp is converted into a revision string.
Shared Access Multiple different processes must be able to access the data in read/write mode. SyncEvolution runs data access backends inside the sync daemon or application. If data can only be accessed inside a single process, SyncEvolution cannot synchronize it. Some kind of remote data access stub inside that process would have to be implemented. libecal and libebook provide access to data stored in EDS. All processes in the system have equal access to the files.
Notifications about data changes When one process is permanently running, it should get notified by the storage system when some other process changes data. Without it, each running process must poll the storage system regularly, wasting power and/or increasing latencies. SyncEvolution currently does not use this, but will need it for automatic syncs that are triggered by local data changes. Provided by "views". Possible via file and directory notifications, not used at the moment.
Safe Concurrent Access When multiple process access the data in read/write mode, some locking mechanism must ensure that no data is lost, for example by overwriting a modified item with an older update or removing a modified item. Without such a mechanism, editing PIM data in a GUI and synchronizing it should better be avoided to be on the safe side. Not supported. API extension patch based on revision strings is pending review. File locking could be used for this. But because all users of the files would have to agree on the same method (which is not the case), the SyncEvolution backend doesn't implement it either.
Efficient and atomic listing of items Before starting a sync, SyncEvolution needs to know whether data is available locally, without actually needing the data. Then revision string based change tracking also needs the revision string of each item. Without a suitable API, SyncEvolution has to fall back to reading all items including their data, which is slower. Without an atomic implementation of this functionality, there is a race condition which can lead to certain changes not getting synchronized or lost. Not supported. The complete item data has to be read to extract REV and LAST-MODIFIED. Done by just reading the directory. Not atomic, though.
Restore item with specific revision string SyncEvolution can restore items to a former revision. Doing so with the original revision string has the advantage that peers who have that revision do not get an incorrect item change sent to them the next time the sync. Minor problem. When syncing with one peer, SyncEvolution works around the lack of the feature by updating the ID/rev string mapping of that peer. Restoring the original revision string is not implemented at all. Not supported. API extension patch is pending review. Could be done via utime(), not implemented yet.