#end dircategory

ASynK User Manual
*****************

This file documents ``ASynK - Awesome Sync (by Karra)''

Copyright (C) 2011, 2012, 2013, 2014, 2015 Sriram Karra <karra.etc@gmail.com>

     Permission is granted to copy, distribute and/or modify this document under the terms of the GNU Free Documentation License, Version 1.3; with the Invariant Section being ``GNU GENERAL PUBLIC LICENSE,'' A copy of the license is included in the section entitled ``GNU Free Documentation License.''

Introduction
************

ASynK is a flexible Contacts synchronization program that works with a variety of Contacts software such as Microsoft Outlook, Microsoft Exchange Server, Google Contacts, CardDAV and Emacs BBDB.

If you would not read through this documentation from cover to cover, but rather learn by looking at examples, head straight to the section titled ``Recipes''. It contains enough examples to help you get going.

So, where were we? Ah, yes, introducing ASynK. Here are some of the high level features of ASynK:

     [file intro.texi not included]

Getting ASynK and Installation
******************************

The official website for ASynK is at: http://asynk.io. You will find links to latest downloads, complete and latest documentaiton, announcements and other information on that website.

The source code for ASynK is hosted on Github at: https://github.com/skarra/AsynK (earlier hosted at: https://github.com/skarra/Gout) Visit the site to grab the bleeding edge development version, or to register an issue you have identified.

Pre-packaged Bundle
===================

You can download the latest version by visiting the downloads section of the official website at http://asynk.io/downloads/.

From Source
===========

You can clone the git repository by using a command such as git clone git://github.com/skarra/ASynK.git. Then run asynk.py as per instructions given in the Using ASynK, section below.

Dependencies / Notes
====================

General Dependencies
--------------------

 1. Python 2.7. You can download Python from: http://www.python.org/download/releases/

 2. If you want to build the documentation yourself you will need some additional tools:

      1. makeinfo: Is needed to build documentation in Info, docbook and xml formats. Version 4.8 is known to work with our sources.

      2. texi2html: Is needed to build documentation in html format. Version 1 .70 is known to work with our sources.

      3. texi2pdf: Is needed to build the documentation in PDF format. Version 1.34 is known to work with our sources.

Microsoft Exchange Specific Notes
---------------------------------

If you want to synchronize to your Exchange Server you could do that from any machine or operating system that can access your Exchange server over the internet. You will need to ask  your system administrators for:

 1. Your full email address

 2. Your account password

 3. The exchange mailbox url

Windows / Outlook Specific Notes
--------------------------------

If you have contacts on your own machine inside a Outlook PST file you can sycn them too. But there are a few additional dependencies:

 1. Microsoft Windows: You must run ASynK on a Windows machine. Any recent version such as XP onwards will do.

 2. MS Outlook: Outlook should be installed. Direct read/write of PST files is not possible. Any recent version should work. ASynK has been reported to work with Office 2007, Office 2008, Office 2010, Office 2013

 3. Pywin32 for Python 2.7: It is available for download http://sourceforge.net/projects/pywin32/files/pywin32

CardDAV Specific Notes
----------------------

Keep in mind that unlike Google Contacts, CardDAV servers do not support a roll back feature. This means any ASynK bugs could potentially lead to loss of information for you. While there have been no complaints of ASynK eating up people's data, please take your own precautions before you deploy this on your live data. The use of the --user-dir= option is also high encouraged.

At the moment the following fields are synched in both ways. More may be supported over time.  Important: Fields not mentioned here will be lost in transit!

 1. First name

 2. Last name

 3. Middle name

 4. Formatted name

 5. Suffix, Prefix

 6. Gender

 7. Email addresses

 8. Phone numbers

 9. Title

 10. Department

 11. Company name

 12. Notes

Specifically, the following are not synched:

 1. IM

 2. WWW

 3. Postal Addresses

 4. Custom fields

BBDB Specific Notes
-------------------

ASynK can parse and write BBDB files without the need for Emacs or BBDB. However, ensure you are NOT using BBDB at the time ASynK is reading/writing the database. Bad things can happen if you are not careful with this. Furthermore, note the following:

 1. BBDB V3: ASynK works best if you use BBDBV3. V3 is the future of BBDB and a significant rewrite of large portions of the code base, but is decidedly not backward compatible with most user customization. You can download it at http://savannah.nongnu.org/projects/bbdb.

     Importantly for ASynK, the file format used by V3 provides for granular time stamps with timezone information and ASynK will just work ``out of the box''. There are also other important changes to the structure of the BBDB record in file format version 7.

 2. Timestamp Customization: ASynK expects timestamps to be in UTC - i.e. they should all end with the +0000. If you have customized your BBDB timestamp format, it can lead to confusing results.

 3. BBDB-2.x: Starting release v0.2.1 ASynK has added limited support for BBDB file format version 6 which is the latest version used by the 2.x releases of BBDB. This version of BBDB is not actively maintained, but is more widely deployed because of its age and availability on XEmacs.

     Support is limited in the sense that the native ver 6 file format timestamps only store year, month and day. So BBDB will overwrite the granular ASynK timestamps with a yyyy-mm-dd timestamp, and your bi-directional sync will not quite work as you would like.

     The above limitation can be overcome by applying the following simple patch. With this patch in place ASynK will work quite well. There is a downside, however - certain time-based search in BBDB may behave differently (search newer, search older etc. will behave differently owing to differences in timestamp format). But one would like to think the ability to do bidirectional sync outweighs this minor irritation.

     commit eb6ba8d342a46d61c45b0ce4c1de7711ff011466
     Author: Sriram Karra <karra.etc@gmail.com>
     Date:   Wed May 16 18:09:50 2012 +0530

         Include UTC time as part of timestamp

         This is a feature in file format 7 available with BBDB v3. However
         that is not available with the 2.x series. A detailed timestamp
         is needed to do a sync with a third party.

         The ideal way would have been to fully implement the version 7
         changes on 2.36. However that involves a lot of changes and is
         risky - for e.g. introducing a new key name field (affix)
         will stretch my understanding of BBDB internals. Best to just
         provide a quick hack for those willing to live with it only to
         try ASynK or similar sync software.

         The obvious pitfall is that commands that work on the timestamp
         field (like newer or older filters and delete) will start
         to behave differently because a string comparison is done to
         check if a date is newer or older.

            *** You have been warned ***

     diff --git a/lisp/bbdb-hooks.el b/lisp/bbdb-hooks.el
     index 01de4f2..b9f0d9d 100644
     --- a/lisp/bbdb-hooks.el
     +++ b/lisp/bbdb-hooks.el
     @@ -59,7 +59,7 @@
      (defvar mh-show-buffer)

     -(defvar bbdb-time-internal-format "%Y-%m-%d"
     +(defvar bbdb-time-internal-format "%Y-%m-%d %T %z"
        "The internal date format.")

      ;;;###autoload
     @@ -69,7 +69,7 @@ for the given record which contains the time when it was last modified.  If
      there is such a field there already, it is changed, otherwise it is added."
        (bbdb-record-putprop record 'timestamp (format-time-string
                            bbdb-time-internal-format
     -                      (current-time))))
     +                      (current-time) t)))

      ;;;###autoload
      (defun bbdb-creation-date-hook (record)
     @@ -78,7 +78,7 @@ which is the current time string."
        ;; hey buddy, we've known about your antics since the eighties...
        (bbdb-record-putprop record 'creation-date (format-time-string
                                bbdb-time-internal-format
     -                          (current-time))))
     +                          (current-time) t)))

      ;;; Determining whether to create a record based on the content of the
     diff --git a/lisp/bbdb.el b/lisp/bbdb.el
     index cc12045..dabbf05 100644
     --- a/lisp/bbdb.el
     +++ b/lisp/bbdb.el
     @@ -1214,7 +1214,7 @@ from string."

      ;;; Address formatting.

     -(defcustom bbdb-time-display-format "%d %b %Y"
     +(defcustom bbdb-time-display-format "%d %b %Y %T %z"
        "The format for the timestamp to be used in the creation-date and
      timestamp fields.  See the documentation for `format-time-string'."
        :group 'bbdb :type 'string)
     @@ -1223,11 +1223,13 @@ timestamp fields.  See the documentation for `format-time-string'."
        "Convert a date from the BBDB internal format to the format
      determined by FORMAT (or `bbdb-time-display-format' if FORMAT not
      present).  Returns a string containing the date in the new format."
     -  (let ((parts (bbdb-split date "-")))
     -    (format-time-string (or format bbdb-time-display-format)
     -                        (encode-time 0 0 0 (string-to-number (caddr parts))
     -                                     (string-to-number (cadr parts))
     -                                     (string-to-number (car parts))))))
     +  (format-time-string (or format bbdb-time-display-format)
     +		      (condition-case err
     +			  (date-to-time date)
     +			(error nil
     +			       (date-to-time (concat date " 00:00:00
     +			       +0000"))))
     +		      t))

      (defalias 'bbdb-format-record-timestamp 'bbdb-time-convert)
      (defalias 'bbdb-format-record-creation-date 'bbdb-time-convert)

Building & Installing Documentation
===================================

The user documentation is in the directory called doc/ in the root of the ASynK source tree. The main source file is asynk.texi in TexInfo format. HTML, PDF and Info files are available on the project home page at: http://asynk.io

If you wish to build the documentation from source, you can try the Makefile in the documentation directory, like so:

   $ cd doc
   $ make pdf
   $ make       # Builds all the targets
   $ make clean
   $ make distclean

The supported doc types are: 'pdf', 'html', 'info', 'docbook', and 'text'. You may have to install additional programs like makeinfo, texi2html etc. See the "Dependencies" section above.

Concepts
********

This chapter explains some of the key terminology we will use through the rest of this documentation. This is also required reading before you can dig into the code, if you are so inclined.

Your Information and How it is Organized
========================================

PIMDBs
------

Google, MS Outlook and BBDB are know as ``PIM Databases'', and each is a ``PIM DB'', or pimdb. PIM stands for Personal Information Manager. PIMDB is PIM Database.

Each PIMDB is referred to using a unique two letter specifier -'bb' for BBDB, 'gc' for Google Contacts, 'cd' for CardDAV and 'ol' for Outlook. This two letter specifier is called a dbid

Items
-----

Your personal information collection includes contacts, notes, appointments, and tasks. In AsynK we refer to each piece of PIM information in your databases as an Item. When we want to refer a specific type of Item we would refer to them as Items - Contact, Task, Note, or Appointment as applicable.

Every Item has an ID known as a ItemID that is guaranteed to be unique within its containing Folder.

A pimdb can contain one or more Item types. Outlook supports all of them. Google covers Contacts and Appointments well, and BBDB only does contacts. In future we will be able to deal with all the contact types, but for now, the only support is for Contacts.

Message Store
-------------

Instead of defining a message store in general terms, let me just give you what they are for each of the PIMDBs we have.

In Google, a Message Store is identified by a user login/password. If you have two Google accounts you have access to two Message Stores on Google Contacts.

With CardDAV, as you can have any number server and a user login/password for your account on that server. In this case a Message Store is the server name - with a leading http or https, and a trailing port number, and associated user credentials.

With MS Outlook, there is a built in concept of a MessageStore. Each account you create becomes a Message Store. If you add a PST file to your profile, it becomes a distinct Message Store. Essentially, The ASynK Message Store corresponds directly with a Outlook Message Store.

A Note on Outlook Profiles: ASynK only works on the default Outlook Profile, and the contained Stores, folders, and contacts. This is important to keep in mind. You cannot specify a different Outlook profile to operate on.

And finally, in BBDB, a BBDB file is the Message Store. Simple :)

Folders
-------

Message Stores are logically divided into Folders. Google and Outlook internally organize their items into Folders (Google actually calls them Groups), but there is no native folder support in BBDB.

Folders are assigned a unique ID determined by the PIMDB, and typically when the folder is created. ASynK provides a way of viewing the available folders, their names, and IDs in a given PIMDB / Message Store.

Folders in BBDB
...............

The fact that there is no native folder support in BBDB is more than offset by the fact that is infinitely customizable, and we can support a user-level model of folders as we wish. This is how it works:

Normally, all the contacts in a BBDB store will be in a folder called default. If, however, a contact has a notes field with key titled folder, it will be put into the logical folder with the name specified in the value part of that BBDB notes. For e.g. if (``folder'' . ``Buddies'') is a notes field in a contact, the ASynK will treat it as being in the folder called Buddies.

If you do not like the folder key for your BBDB notes field, and want to call it group instead, you can do that too. Refer to the Customizing AsynK chapter below on how this can be done.

A BBDB folder's Name and ID are the same, and are taken from the value of the notes field as explained above.

ASynK Profile
=============

ASynK allows you to keep any folder in a pimdb in sync with any folder in one of the other pimdbs. The configuration and state of synchronization between a pair of folders is captured in a sync Profile. Most ASynK operations operate on a specified sync profile. So you will need to set up at least one profile before you can play with ASynK.

Each Profile has the following information associated with it:

 1. Name: A unique alphanumeric string that identifies the profile, and is required for most sync operations. This is specified at the time of profile creation, and has to be provided by the user. A profile's name cannot be changed after creation.

 2. Folder IDs: The IDs for the two folders that are to be synched. The required IDs have to be specified by the user at the time of profile creation. The IDs are assigned by the PIM DBs when the folders are created. ASynK can be used to list the existing folders in a PIMDB and their IDs, so the user can use the appropriate IDs at profile creation time.

 3. DB IDs: Two DB IDs that hold the folders that are to be synched. This is specified at the time of profile creation and has to be specified by the user. They have to one of the supported DB IDs, which are 'bb', 'gc', 'cd' and 'ol', in this version. The order in which the DB IDs are specified matters - as explained below.

 4. Sync Timestamps: The start and end of the last successful sync operation for this profile.  The end of the last sync is the timestamp used to query the pimdbs for updates that have to be synched. These are initialized and updated automatically by the system and you do not have to worry about it.

 5. Conflict Resolution Direction: Occasionally one ends up editing an item in multiple places or devices, and this situation leads to a conflict - i.e. the same item has been updated since the last sync on both ends. There is no magic way to figure out which is the correct version. So we have to use a pre-determined tie breaker. This parameter tells the system in which direction to resolve this conflict. This field can be specified at the time of profile creation, but defaults to the first dbid specified in the profile. Once profile is created, the direction of conflict resolution cannot be specified for a operation.

 6. Sync Direction: Although ASynK is designed to be a two way sync, you might be interested in setting up certain profiles to sync in only one direction - i.e. new entries and updates to existing entries will only flow in one direction. This can be specified at the time of profile creation and also be specified for every sync operation. If it is a one way sync, currently the flow will be from db1 to db2 (i.e. the first db in the profile to the second db). Note: This has to be fixed.

Using ASynK
***********

Recent releases of ASynK (v0.2.0+) only support a command line user interface - even on Windows. Once you have your platform-specific dependencies installed, there is no difference at all in the way ASynK is invoked on any platform.

There are plans to develop a browser-based GUI, but work on that has not started yet.

Invoking Help
=============

As of the latest version, only a command line interface is available. There are plans to provide a browser based UI at a later stage.

To run ASynK, change to the root of the downloaded ASynK directory, and type python asynk.py -h to get a detailed help screen as show below. In the following sections we look at each of the flags and options in detail (in time, of course).
Srirams-MacBook-Pro:asynk.co sriramkarra$ ./asynk.py --help
usage: asynk.py [-h] [--dry-run] [--sync-all]
                [--op {list-folders,create-folder,create-store,show-folder,del-folder,list-profiles,list-profile-names,find-profile,create-profile,show-profile,del-profile,sync,clear-sync-artifacts}]
                [--user-dir USER_DIR]
                [--db {bb,gc,ol,cd,ex} [{bb,gc,ol,cd,ex} ...]]
                [--store STORE [STORE ...]] [--folder FOLDER [FOLDER ...]]
                [--item ITEM] [--name NAME] [--direction {1way,2way}]
                [--label-regex LABEL_REGEX]
                [--conflict-resolve CONFLICT_RESOLVE]
                [--gcpwd GCPWD [GCPWD ...]] [--cduser CDUSER [CDUSER ...]]
                [--cdpwd CDPWD [CDPWD ...]]
                [--log {debug,info,error,critical}] [--version]

ASynK: PIM Awesome Sync by Karra

optional arguments:
  -h, --help            show this help message and exit
  --dry-run             Do not sync, but merely show what will happen if a
                        sync is performed.
  --sync-all            when used with --op=sync, this will ignore previous
                        synchronization state, and perform a complete resync.
  --op {list-folders,create-folder,create-store,show-folder,del-folder,list-profiles,list-profile-names,find-profile,create-profile,show-profile,del-profile,sync,clear-sync-artifacts}
                        Specific management operation to be performed.
  --user-dir USER_DIR   Directory to store ASynK config files, logs directory,
                        BBDB backups directory, etc.
  --db {bb,gc,ol,cd,ex} [{bb,gc,ol,cd,ex} ...]
                        DB IDs required for most actions. Some actions need
                        two DB IDs - do it with two --db flags. When doing so
                        remember that order might be important for certain
                        operations.
  --store STORE [STORE ...]
                        Specifies store ID(s) to be operated on.
  --folder FOLDER [FOLDER ...]
                        For operations that need folder ids, this option
                        specifies them. More than one can be specified
                        separated by spaces
  --item ITEM           For Item operations specify the ID of the Item to
                        operate on.
  --name NAME           For profile operations, specifies profile name. For
                        Folder operations, specifies folder name
  --direction {1way,2way}
                        Specifies whether a sync has to be unidirectional or
                        bidirectional. Defaults to bidirectional sync, i.e.
                        "2way"
  --label-regex LABEL_REGEX
                        A regular expression for sync artification to be
                        cleared from specified folder. This is to be used
                        independently of any sync profile.
  --conflict-resolve CONFLICT_RESOLVE
                        Specifies how to deal with conflicts in case of a
                        bidirectional sync and an item is modified in both
                        places. It should be set to 1 or 2 to specify the one
                        to be used; in case the dbs are unique. For e.g. if
                        you are synching from BBDB to Google Contacts, then
                        you can also specify the dbid itself (i.e. bb or gc)
  --log {debug,info,error,critical}
                        Specify level of console logging. Note that DEBUG
                        level logs are always written to a log file for
                        tracking purposes
  --version             show program's version number and exit

Google Authentication:
  --gcpwd GCPWD [GCPWD ...]
                        Google password. Relevant only if --db=gc is used. You
                        can specify two if you are operating with 2 gc dbs.
                        You could also specify one from netrc and one on the
                        command line. First one can optionally be "None"
                        (without the quotes). If this option is not specified,
                        user is prompted password from stdin as required.

CardDAV Server Authentication:
  --cduser CDUSER [CDUSER ...]
                        CardDAV username. Relevant only if --db=cd is used.
                        You can specify two if you are operating with 2 cd
                        dbs. You could also specify one from netrc and one on
                        the command line. First one can optionally be "None"
                        (without the quotes). If this option is not specified,
                        user is prompted for it from stdin if required.
  --cdpwd CDPWD [CDPWD ...]
                        CardDAV password. Relevant only if --db=cd is used.
                        You can specify two if you are operating with 2 cd
                        dbs. You could also specify one from netrc and one on
                        the command line. First one can optionally be "None"
                        (without the quotes). If this option is not specified,
                        user is prompted for it from stdin if required.

TODO: Detailed documentation on what each of the above flags mean, etc.

Customization
*************

ASynK differentiates between application configuration and application state. Configuration is user-facing key-value information, and something that a user does not typically change in the middle of a run. Configuration tells ASynK what to sync, where to put log files, etc.

Application state, on the other hand, is not meant to be modified directly by the user. It is used internally by the program to manage various things about the sync, etc.

In this section we only talk about application configuration. The application state is explained in more detail in the Internals, section.

config.json and config.py
=========================

The entire application configuration is stored as a JSON file in the application_root/config/ directory. It has copious comments and explanation of its various fields, so we will only list the main configurable parameters here, and will just point you to the source to find out more. You should not edit this JSON file directly as it might get updated in subsquent releases of ASynK.

All of your own customizations should be placed in the file ~/.asynk/config.py. You can look in the file for example customizations and further instructions. The first time you run ASynK, a copy of a sample config.py is made to ~/.asynk (which is the default per-user asynk config directory), or to a directory specified with the --user-dir command line option. For all subsequent runs, the default config.json is loaded, then the user customizations are executed by loading the config.py file from the user directory.

 1. Location of directory containing logs. This should be the name of a directory relative to the ASynK user directory, which is ~/.asynk/ by default.

 2. The prefix and separator to use for the ASynK sync tags/labels.

 3. Default values for newly created profile, like sync direction, etc.

 4. DB specific configuration. There are many instances where the capabilities of one PIM DB does no match another's. If care is not take to bridge this gap, synching back and forth could lead to loss of information. ASynK is designed to ensure this does not happen, and we try very hard.

     Bridging this gap sometimes means taking a call on how to handle a particular situation. We have tried to make these essentially arbitrary calls configurable so interested users can tweak this to their liking.

      1. Outlook

           1. The fields to be synched: Outlook has a gazillion properties, many of them seemingly redundant. We have identified a subset of the really relevant ones here, and you can add or remove stuff you want or don't want.

           2. Email classification by domains: Outlook does not really classify email addresses into Work, Home, Other etc. like Google does. Here you can tell ASynK how to do that based on the email addresses.

      2. BBDB

           1. Location of directory containing backup of BBDB stores. This should the name of a directory relative to the asynk user directory, which is ~/.asynk/ by default.

           2. Text Encodings: The list of text encodings that you want ASynK to try while attempting to read your BBDB

           3. Email classification by domains: Same as explained for Outlook

           4. Postal address map: Same as above, except that this is for postal addresses.

           5. Phones map: Ditto, for phones

           6. Notes map: BBDB really is really flexible, to the point that very basic fields are not standardized. There is no date of birth, middle name, web url, gender, and similar fields. However one can always have such data as a user-defined field. This map allows the mapping from a bbdb field name to one of the standard fields.

      3. Google Contacts

           1. Postal Map: Google's address system is interesting - a user can either have a standard category, or a user defined label. A generic user defined label has the same problem as with others - it is hard to classify. So we need a map for Google as well.

     Note that your sync field customizations for all the databases should be consistent, otherwise there would be information loss as you sync your contacts around.

Authentication using ~/.netrc
=============================

For Google and CardDAV you are required to provide a username and password to perform most operations. These credentials can be specified in a number of ways including command line flags, or through the STDIN on being prompted for them. There is yet another way to set this up that will reduce your typing and will consolidate all your asynk related authentication information in the same place. That is by using the ~/.netrc file.

Let's say you have a ASynK profile named testbbcc connects your BBDB file to a Google Contacts account. You can specify the auth credentials for this profile by inserting the following lines in youru ~/.netrc.

machine gc_testbbgc
  login <<< username >>>
  password <<< password >>>

Here the gc_ prefix to the profile name specifies clearly that this is a Google account. Replace the <<< >>> with your actual values.

Similarly for a CardDAV account credentials the gc_ prefix will become cd_. Like so:

machine cd_testbbcd
  login <<< username >>>
  password <<< password >>>

Using netrc for like-pair synching
----------------------------------

If your profile is set up to sync betwen two like stores, for e.g. between two Google Contacts accounts, you can specify the login credentials in your netrc as follows:

machine gc1_testgcgc
  login <<< username >>>
  password <<< password >>>

machine gc2_testgcgc
  login <<< username >>>
  password <<< password >>>

Note the name of the machine specifies if it belongs to the first or second collection. Note that you could use the same credentials for both collections. This case would be useful to copy / mirror contacts from one folder to another.

Recipes
*******

Note to Self: This section ideally should only give examples without really explaining the options, constraints  etc. in depth - that should be done in the previous section on ``Using ASynK'' where the complete documentation for every flag in the system will be documented, and will the authoritative single reference for such things. This will entail some reorg of the doc content and much additional content as well...

The easiest way to get used to AsynK is to look at some examples. This chapter contains some of the simplest use cases. You can refer to section Using ASynK, for detailed documentation on each flag, legal and illegal options, etc.

Basic Usage: using the default sync profiles
============================================

$ ./asynk.py --op=sync [--dry-run] [--log=debug]

ASynK comes with two default profiles to help you get a quick feel for the sync features.

When you run the above mentioned command on a windows box a default profile called defolbb kicks in and contacts in your  ~/.bbdb file are bi-directionally synced to your default Outlook folder.

On any non-windows platform, your ~/.bbdb is merely ``synched'' to another BBDB file in /tmp/asynk.bbdb. The default profile that kicks in is defbbbb

Note: In earlier versions the default profiles helped you sync easily to your Google account, but due to some changes that Google made to its authentication flow, a simple no-frills sync is no longer possible. Setting up Google sync is covered elsewhere in this documentation.

Explore Folders and Create Synk Profile
=======================================

 1. Determine the PIMDBs you want to sync - i.e. select two from bb, gc, and ol

 2. Determine the Message Stores you want to sync - remember that for Google, a message store is an account with login credentials, for Outlook it maps to an Outlook MessageStore (IDs are obtained from a --op=list-folders command shown below), and for BBDB it is a file - specified by a filename. The Stores are specified with the --store=<storeid, storeid> option.

 3. List the folders you have in your Message Stores to identify the folders that actually contain the Items you want to synchronize.
     Cardassia:Gout sriramkarra$ python asynk.py --op=list-folders --db gc --store karra.etc < login.auth
     [18:08:08.216     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-09.18-08-08.216602
     [18:08:08.572     INFO] Logging into Google...
     [18:08:10.603     INFO] Listing all folders in PIMDB gc...
     [18:08:11.797     INFO]   0: Contacts Name: System Group: My Contacts ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/6
     [18:08:11.797     INFO]   1: Contacts Name: System Group: Friends     ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/d
     [18:08:11.797     INFO]   2: Contacts Name: System Group: Family      ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/e
     [18:08:11.797     INFO]   3: Contacts Name: System Group: Coworkers   ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/f
     [18:08:11.797     INFO]   4: Contacts Name: TestBB                    ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/2fae451c0fe771d5
     [18:08:11.797     INFO]   5: Contacts Name: Gout                      ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/41baff770f898d85
     [18:08:11.798     INFO]   6: Contacts Name: Starred in Android        ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/5254f4a808c5e7d8
     [18:08:11.798     INFO] Listing all folders in PIMDB gc...done

     The output should be self explanatory. The group ID is the full URI in the last column of output. A few things to note:

      1. For a google sync profile operation, if a --store option is skipped, the user is prompted for a username and password from stdin. If a --store option is given on the command line, it is used as the login name and only the password is prompted for.

      2. In the above example, we have stored the password (in plain text) in a file called ``login.auth'' and piping it to ASynK.

     Cardassia:Gout sriramkarra$ ./asynk.py --op=list-folders --db bb --store test/bbdb.utah.ver7
     [18:27:07.147     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-09.18-27-07.147160
     [18:27:07.149     INFO] Parsing BBDB file /Users/sriramkarra/stuff/code/python/Gout/test/bbdb.utah.ver7...
     [18:27:07.396     INFO] Successfully parsed 338 entries.
     [18:27:07.396     INFO] Listing all folders in PIMDB bb...
     [18:27:07.396     INFO]   1: Contacts.	Name: default;	GID: default;
     [18:27:07.396     INFO] Listing all folders in PIMDB bb...done

     Here we passed the relative path of a BBDB file to list-folders. Recall that for BBBD the message store is the full BBDB file. In this instance the store is a typical BBDB database file where no contacts have the folder notes field, hence all of them are put in the ``default'' folder. Note how the name and ID of the folder are the same.
     TODO: Insert a sample Outlook list-folders

 4. Optionally you would like to create a folder in one of the Message Stores to hold your synchronized contacts. While you can create new folders in the native UI of Google Contacts or Outlook and use them with ASynK, we just provide a convenience option of creating one from the command line.
     Cardassia:Gout sriramkarra$ ./asynk.py --op create-folder --name 'Demo' --db gc --store karra.etc@gmail.com < login.auth
     [22:30:00.827     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-12.22-30-00.827089
     [22:30:00.829     INFO] Logging into Google...
     [22:30:02.871     INFO] Successfully created group. ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/6c2034580ea2ac15

     Cardassia:Gout sriramkarra$ ./asynk.py --op=list-folders --db gc --store karra.etc < login.auth
     [22:30:08.870     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-12.22-30-08.870610
     [22:30:08.872     INFO] Logging into Google...
     [22:30:09.899     INFO] Listing all folders in PIMDB gc...
     [22:30:10.331     INFO]   0: Contacts Name: System Group: My Contacts ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/6
     [22:30:10.332     INFO]   1: Contacts Name: System Group: Friends     ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/d
     [22:30:10.332     INFO]   2: Contacts Name: System Group: Family      ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/e
     [22:30:10.332     INFO]   3: Contacts Name: System Group: Coworkers   ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/f
     [22:30:10.332     INFO]   4: Contacts Name: TestBB                    ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/2fae451c0fe771d5
     [22:30:10.332     INFO]   5: Contacts Name: Gout                      ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/41baff770f898d85
     [22:30:10.333     INFO]   6: Contacts Name: Starred in Android        ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/5254f4a808c5e7d8
     [22:30:10.333     INFO]   7: Contacts Name: Demo                      ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/6c2034580ea2ac15
     [22:30:10.333     INFO]   8: Contacts Name: For Testing OLGC          ID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/727ac1f9891121f7

     As we can see, the new folder has been created successfully.

 5. List your sync profiles. When you first start out, there is only a dummy sync profile, whose only purpose for existence is so you can do a list-profiles with your shiny new ASynK download :-) So let's do it.
     Creating default state.json to hold application status
     [22:35:00.058     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/ASynK/logs/asynk_logs.2012-05-12.22-35-00.058672
     [22:35:00.060     INFO]
     [22:35:00.060     INFO] *** Profile   : sample ***
     [22:35:00.061     INFO]   Collection 1:
     [22:35:00.061     INFO]     DB ID     : gc
     [22:35:00.061     INFO]     Store ID  : karra.etc@gmail.com
     [22:35:00.061     INFO]     Folder ID : http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/2fae451c0fe771d5
     [22:35:00.062     INFO]   Collection 2:
     [22:35:00.062     INFO]     DB ID     : bb
     [22:35:00.062     INFO]     Store ID  : test/bbdb.utah.ver7
     [22:35:00.062     INFO]     Folder ID : default
     [22:35:00.062     INFO]   sync_start  : 1980-01-01T00:00:00.00+00:00
     [22:35:00.062     INFO]   sync_stop   : 1980-01-01T00:00:00.00+00:00
     [22:35:00.063     INFO]   sync_dir    : SYNC2WAY
     [22:35:00.063     INFO]   confl_res   : gc
     [22:35:00.063     INFO]   olgid       : 0x9001

     The output should be fairly self explanatory. The sample profile is between a GC folder and the default folder in a specific BBDB store. It also specifies that the sync should be two way, and You do not have to understand what ``olgid'' means. This is

 6. You would create up a sync profile that specifies the various details ASynK needs to know to perform a sync. (See section ASynK Profile). From here on your regular sync is done through a reference to the profile name, and you do not have to type out all the confusing IDs etc. every time.
     Cardassia:Gout sriramkarra$ python asynk.py --op=create-profile --db gc bb  \
     --folder 'http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/2fae451c0fe771d5' \
     'default' --store karra.etc 'test/bbdb.olbb' --name etcbb3
     [00:33:39.347     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-13.00-33-39.347515
     [00:33:39.373     INFO] Successfully added profile: etcbb3

     The --op=create-profile operation takes two dbids, two store ids and two folder ids. The command line is quite a handful, but you should not have to repeat this operation too often. It is worth noting the following about the options passed:

       * The order of values for the --db, --store and --folder options is important and should be consistent - i.e. the first value from each of these three will be used as one collection, and second values are used as

       * If "gc" is one of the dbids, and store id is missing, it will be picked up from stdin.

       * If BBDB is one of the dbids and folder id is missing it is assumed to be the default folder.

       * If ol is one of the dbids and the store id is missing, we use the default message store in the default profile

     Cardassia:Gout sriramkarra$ python asynk.py --op=list-profiles
     [00:34:44.785     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-13.00-34-44.785442
     [00:34:44.787     INFO]
     [00:34:44.790     INFO] *** Profile   : etcbb3 ***
     [00:34:44.790     INFO]   Collection 1:
     [00:34:44.790     INFO]     DB ID     : gc
     [00:34:44.790     INFO]     Store ID  : karra.etc
     [00:34:44.790     INFO]     Folder ID : http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/2fae451c0fe771d5
     [00:34:44.791     INFO]   Collection 2:
     [00:34:44.791     INFO]     DB ID     : bb
     [00:34:44.791     INFO]     Store ID  : test/bbdb.olbb
     [00:34:44.791     INFO]     Folder ID : default
     [00:34:44.791     INFO]   sync_start  : 1980-01-01T00:00:00.00+00:00
     [00:34:44.791     INFO]   sync_stop   : 1980-01-01T00:00:00.00+00:00
     [00:34:44.791     INFO]   sync_dir    : SYNC2WAY
     [00:34:44.791     INFO]   confl_res   : gc
     [00:34:44.791     INFO]   olgid       : 0x0
     [00:34:44.792     INFO]
     [00:34:44.792     INFO] *** Profile   : sample ***
     [00:34:44.792     INFO]   Collection 1:
     [00:34:44.792     INFO]     DB ID     : gc
     [00:34:44.792     INFO]     Store ID  : karra.etc@gmail.com
     [00:34:44.792     INFO]     Folder ID : http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/3b83f70e88fa08bd
     [00:34:44.792     INFO]   Collection 2:
     [00:34:44.792     INFO]     DB ID     : ol
     [00:34:44.792     INFO]     Store ID  : None
     [00:34:44.792     INFO]     Folder ID : AAAAADWE5+lnNclLmn8GpZUD04eCRRoA
     [00:34:44.792     INFO]   sync_start  : 1980-01-01T00:00:00.00+00:00
     [00:34:44.793     INFO]   sync_stop   : 1980-01-01T00:00:00.00+00:00
     [00:34:44.793     INFO]   sync_dir    : SYNC2WAY
     [00:34:44.793     INFO]   confl_res   : gc
     [00:34:44.793     INFO]   olgid       : 0x9001

     And as we can confirm the new profile is now set up and available for use. Please note that there is not much error checking at the time of profile creation. For e.g. it is the user's responsibility to ensure there is a specified folder and store in a given pimdb, etc. If there are any errors, they will show up in subsequent sync operations.

Sync Actions and Related Cleanup
================================

 1. Perform a ``dry run'' to get a sense of how many contacts are going to get copied from where to where.
     Cardassia:Gout sriramkarra$ python asynk.py --op=sync --dry-run --name etcbb2 < login.auth
     [00:53:10.643     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-13.00-53-10.643187
     [00:53:10.648     INFO] Logging into Google...
     [00:53:12.143     INFO] Parsing BBDB file /Users/sriramkarra/stuff/code/python/Gout/test/bbdb.utah.ver7...
     Password: [00:53:12.291     INFO] Successfully parsed 338 entries.
     [00:53:12.291     INFO] BBDB database not backed up for dry run
     [00:53:12.292     INFO] Last synk for profile etcbb2 was at: 2012-05-11T11:01:56.57Z
     [00:53:12.292     INFO] Querying Google for status of Contact Entries...
     [00:53:13.157     INFO] Response received from Google. Processing...
     [00:53:13.157     INFO] No entries in feed.
     [00:53:13.159     INFO] ==== gc =====
     [00:53:13.160     INFO]    New              :     0
     [00:53:13.160     INFO]    Modified         :     0
     [00:53:13.160     INFO]    Unchanged        :     0
     [00:53:13.160     INFO]                       =====
     [00:53:13.160     INFO]    Total Entries    :     0
     [00:53:13.160     INFO]    Deleted          :     0
     [00:53:13.160     INFO] ==== bb =====
     [00:53:13.161     INFO]    New              :     0
     [00:53:13.161     INFO]    Modified         :     0
     [00:53:13.161     INFO]    Unchanged        :   338
     [00:53:13.161     INFO]                       =====
     [00:53:13.161     INFO]    Total Entries    :   338
     [00:53:13.161     INFO]    Deleted          :     0
     [00:53:13.161     INFO] Number of entries modified both places (conflicts): 0
     [00:53:13.162     INFO] conflict resolve direction : bb. db1id: gc, db2id: bb
     [00:53:13.162     INFO] After conflict resolution, size of gc mod :     0
     [00:53:13.162     INFO] After conflict resolution, size of bb mod :     0

     It's not technically a complete dry run. For now we just get a summary of stats of what will happen during a proper sync. In the above run we see that there are no new entries or modifications at either end. Essentially if you perform a sync now, no entries need to be copied from anywhere to anywhere.

     Let's see one more example, this time with a sync profile containing an Outlook folder.
     Z:\Gout>python asynk.py --op=sync --name realolbb --dry-run
     [11:26:33.790     INFO] Debug logging to file: Z:\Gout\logs\asynk_logs.2012-05-13.11-26-33.791000
     [11:26:33.868     INFO] Parsing BBDB file Z:\Gout\test/bbdb.olbb...
     [11:26:33.930     INFO] Successfully parsed 0 entries.
     [11:26:33.930     INFO] Initalizing MAPI...
     [11:26:34.368     INFO] Initalizing MAPI...done
     [11:26:34.368     INFO] Opening default profile in MAPI...
     [11:26:34.476     INFO] Opening default profile in MAPI...done
     [11:26:34.789     INFO] Looking for PIM folders in message store: Personal Folders...
     [11:26:34.805     INFO] Appointments not supported. Ignoring.
     [11:26:34.944     INFO] Looking for PIM folders in message store: Personal Folders...done
     [11:26:35.256     INFO] Looking for PIM folders in message store: Master Task List...
     [11:26:35.273     INFO] Appointments not supported. Ignoring.
     [11:26:35.273     INFO] Appointments not supported. Ignoring.
     [11:26:35.647     INFO] Looking for PIM folders in message store: Master Task List...done
     [11:26:37.674     INFO] BBDB database not backed up for dry run
     [11:26:37.690     INFO] Last synk for profile realolbb was at: 2012-05-12T02:38:08.34Z
     [11:26:37.690     INFO] Querying MAPI for status of Contact Entries
     [11:26:37.706     INFO] Data obtained from MAPI. Processing...
     [11:26:37.706     INFO] ==== bb =====
     [11:26:37.706     INFO]    New              :     0
     [11:26:37.706     INFO]    Modified         :     0
     [11:26:37.706     INFO]    Unchanged        :     0
     [11:26:37.706     INFO]                       =====
     [11:26:37.706     INFO]    Total Entries    :     0
     [11:26:37.706     INFO]    Deleted          :     0
     [11:26:37.706     INFO] ==== ol =====
     [11:26:37.706     INFO]    New              :     2
     [11:26:37.706     INFO]    Modified         :     0
     [11:26:37.706     INFO]    Unchanged        :     0
     [11:26:37.721     INFO]                       =====
     [11:26:37.721     INFO]    Total Entries    :     2
     [11:26:37.721     INFO]    Deleted          :     0
     [11:26:37.721     INFO] Number of entries modified both places (conflicts): 0
     [11:26:37.721     INFO] conflict resolve direction : bb. db1id: bb, db2id: ol
     [11:26:37.721     INFO] After conflict resolution, size of bb mod :     0
     [11:26:37.721     INFO] After conflict resolution, size of ol mod :     0

 2. Perform the actual sync. Note that we do not provide a --name argument to specify the profile name. ASynK will remember the last profile that was operated on and use it in case a --name is not explicitly specified.
     Z:\Gout>python asynk.py --op=sync
     [11:26:56.769     INFO] Debug logging to file: Z:\Gout\logs\asynk_logs.2012-05-13.11-26-56.769000
     [11:26:56.769     INFO] Parsing BBDB file Z:\Gout\test/bbdb.olbb...
     [11:26:56.769     INFO] Successfully parsed 0 entries.
     [11:26:56.769     INFO] Initalizing MAPI...
     [11:26:57.144     INFO] Initalizing MAPI...done
     [11:26:57.144     INFO] Opening default profile in MAPI...
     [11:26:57.253     INFO] Opening default profile in MAPI...done
     [11:26:57.549     INFO] Looking for PIM folders in message store: Personal Folders...
     [11:26:57.579     INFO] Appointments not supported. Ignoring.
     [11:26:57.782     INFO] Looking for PIM folders in message store: Personal Folders...done
     [11:26:58.016     INFO] Looking for PIM folders in message store: Master Task List...
     [11:26:58.048     INFO] Appointments not supported. Ignoring.
     [11:26:58.063     INFO] Appointments not supported. Ignoring.
     [11:26:58.312     INFO] Looking for PIM folders in message store: Master Task List...done
     [11:27:00.591     INFO] Backedup BBDB Store (Z:\Gout\test/bbdb.olbb) to file: Z:\Gout\./backups\bbdb_backup.realolbb.2012-05-13.
     -27-00.591000
     [11:27:00.684     INFO] Last synk for profile realolbb was at: 2012-05-12T02:38:08.34Z
     [11:27:00.684     INFO] Querying MAPI for status of Contact Entries
     [11:27:00.684     INFO] Data obtained from MAPI. Processing...
     [11:27:00.684     INFO] ==== bb =====
     [11:27:00.684     INFO]    New              :     0
     [11:27:00.684     INFO]    Modified         :     0
     [11:27:00.684     INFO]    Unchanged        :     0
     [11:27:00.684     INFO]                       =====
     [11:27:00.716     INFO]    Total Entries    :     0
     [11:27:00.716     INFO]    Deleted          :     0
     [11:27:00.716     INFO] ==== ol =====
     [11:27:00.716     INFO]    New              :     2
     [11:27:00.716     INFO]    Modified         :     0
     [11:27:00.730     INFO]    Unchanged        :     0
     [11:27:00.730     INFO]                       =====
     [11:27:00.730     INFO]    Total Entries    :     2
     [11:27:00.730     INFO]    Deleted          :     0
     [11:27:00.730     INFO] Number of entries modified both places (conflicts): 0
     [11:27:00.730     INFO] conflict resolve direction : bb. db1id: bb, db2id: ol
     [11:27:00.730     INFO] After conflict resolution, size of bb mod :     0
     [11:27:00.730     INFO] After conflict resolution, size of ol mod :     0
     [11:27:00.730     INFO] =====================================================
     [11:27:00.730     INFO]    Sending New bb entries to ol
     [11:27:00.730     INFO] =====================================================
     [11:27:00.730     INFO] No new entries that need to be synched
     [11:27:00.730     INFO] =====================================================
     [11:27:00.730     INFO]    Sending Modified bb entries to ol
     [11:27:00.746     INFO] =====================================================
     [11:27:00.746     INFO] No modified entries that need to be synched
     [11:27:00.746     INFO] =====================================================
     [11:27:00.746     INFO]    Sending New ol entries to bb
     [11:27:00.746     INFO] =====================================================
     [11:27:00.746     INFO] 2 new entries to be synched.
     [11:27:00.809     INFO] Saving BBDB File Z:\Gout\test/bbdb.olbb...
     [11:27:00.809     INFO] =====================================================
     [11:27:00.996     INFO]    Sending Modified ol entries to bb
     [11:27:00.996     INFO] =====================================================
     [11:27:00.996     INFO] No modified entries that need to be synched

     We see that the output from a dry-run operation is repeated, and then the actual two way sync happens. Note the following:

      1. The BBDB file is backed up before any modifications are made. The location is echoed to the console as an INFO message. It is also captured in a log file as noted.

      2. If there are *any* errors during a sync operation, the timestamps are not updated, however some of the database entries may well have been made persistent, and it is not possible to roll back. In database terms, ASynK does not do transactional writes and roll backs. This could create problems if there are too many failures. It's a lot of work to change this behaviour. So let's study the problem in the field and take a call on how to tackle it.

Synching to CardDAV
===================

Setting up a profile to sync with a CardDAV profile may be a bit confusing at best, and downright intimidating at worst. Part of the problem is the Store ID itself is a URL, and it is not a standard - varying from one software to another.

A reminder - CardDAV support is the newest feature in ASynK, and there may be some rough edges. Also most CardDAV servers do not allow a roll back - like Gmail does - to recover any lost data. So convince yourself that ASynK will work for you before you start using it.

Creating a profile with CardDAV
-------------------------------

   $ ./asynk.py --op=create-profile --name testbbcd --user-dir=~/.asynk.cd --db bb cd --folder default default --store <opath-to-bbdb-file> <CardDAV URL> --cduser=<username>

Ensure you are able to login to your account properly.
   $ ./asynk.py --op=sync --user-dir=~/.asynk.cd --name testbbcd --dry-run
   $ ./asynk.py --op=sync --user-dir=~/.asynk.cd --name testbbcd

CardDAV URL to use for the --store parameter
............................................

The thing that confuses most people is the CardDAV url which is not in any standard format. Here are some of the known CardDAV servers, and the URL that usually works with them. Replace server.org with the name of your CardDAV server. If your server is not to be found here, a good way to search the internet for the URL to use is to use a software like Addressbook in OS X; the same URL will work with ASynK as well.

 1. Apple CalendarServer: https://server.org:8443/ or http://server.org/

 2. OwnCloud: https://server.org/owncloud/remote.php/carddav

 3. Baikal Server: http://server.org/card.php

Note that for the port number may be optional. If Apple CalendarServer is configured to respond to both http and https the a port number will be required. Check your server documentation.

CardDAV URL to use for the --folder parameter
.............................................

You can just use ``default'' as the folder ID. But if you feel like using a specific folder, note that the foid parameter in your state.json should look like the following for each of the tested CardDAV servers:

 1. Apple CalendarServer: /addressbooks/__uids__/skarrag/addressbook/

 2. OwnCloud: /owncloud/remote.php/carddav/addressbooks/skarra/contacts/

 3. Baikal Server: /card.php/addressbooks/skarra/default/

Synching to Microsoft Exchange
==============================

A reminder - Exchange support is the newest feature in ASynK, and there may be some rough edges. We have taken care to ensure that any deletes from the server move the contacts to the Trash Bin from where you can recover items if something goes wrong. However convince yourself that ASynK works fine before using it with your prodcution data.

Creating a profile with Exchange
--------------------------------

   $ ./asynk.py --op=create-profile --name testbbex --db bb ex --folder default Contacts --store <opath-to-bbdb-file> <Exchange URL>

The Exchange URL is of the form: https://outlook.office365.com/EWS/Exchange.asmx - you should ask your system administrator.

Ensure you are able to login to your account properly.
   $ ./asynk.py --op=sync --user-dir=~/.asynk.cd --name testbbcd --dry-run

If everything works fine, you can try out without the dry-run option.

Misc Housekeeping Activities
============================

 1. Clear Sync Flags from Contact Entries. Recall that ASynK stores some tracking information (such as the contact's IDs in other remote folders) as custom user fields. These custom fields are readily viewable and editable in Google Contacts and BBDB, and as you can manually edit or delete them on a per contact basis, if you so desire (although it is most definitely not recommended). MS Outlook, on the other hand, does not provide any reasonable way to view or manipulate these fields.

     ASynK provides an automated way to clear out sync related tags from a specified folder. It is best explained through a few examples. But before you should know that ASynK tracks IDs using properties with labels like: ``asynk:profile:ol''. There are three parts separated by a colon. The first part is always asynk. The middle portion is the name of the profile. The last portion is the PIMDB ID in whose ID we are tracking.

     The --op=clear-sync-artifacts operation takes a dbid (mandatory), a storeid and a folderid. If the storeid or folderid are missing ASynK tries to use defaults or query the user as per conventions explained earlier (TODO: insert ref). The final argument is the most interesting: it is a regular expression to match against the custom properties of contacts in the specified folder.
     Z:\Gout>python asynk.py --op=clear-sync-artifacts --db bb --store test/bbdb.olbb --label-re="asynk:realobb"
     [10:59:05.456     INFO] Debug logging to file: Z:\Gout\logs\asynk_logs.2012-05-13.10-59-05.456000
     [10:59:05.456     INFO] Parsing BBDB file Z:\Gout\test/bbdb.olbb...
     [10:59:05.456     INFO] Successfully parsed 20 entries.
     [10:59:05.470     INFO] Found 0 contacts with matching sync tag(s).
     [10:59:05.470     INFO] Saving BBDB File Z:\Gout\test/bbdb.olbb...

     In the above example, we try to delete any sync labels for the realolbb profile. The output shows that there were 20 entries in all, but nothing was really cleared out as there were no matches.
     Z:\Gout>python asynk.py --op=clear-sync-artifacts --db ol --folder "AAAAADWE5+lnNclLmn8GpZUD04ciRhoA" --label-re="asynk:"
     [11:08:42.651     INFO] Debug logging to file: Z:\Gout\logs\asynk_logs.2012-05-13.11-08-42.652000
     [11:08:42.667     INFO] Initalizing MAPI...
     [11:08:43.134     INFO] Initalizing MAPI...done
     [11:08:43.134     INFO] Opening default profile in MAPI...
     [11:08:43.275     INFO] Opening default profile in MAPI...done
     [11:08:43.602     INFO] Looking for PIM folders in message store: Personal Folders...
     [11:08:43.618     INFO] Appointments not supported. Ignoring.
     [11:08:43.836     INFO] Looking for PIM folders in message store: Personal Folders...done
     [11:08:44.101     INFO] Looking for PIM folders in message store: Master Task List...
     [11:08:44.101     INFO] Appointments not supported. Ignoring.
     [11:08:44.118     INFO] Appointments not supported. Ignoring.
     [11:08:44.710     INFO] Looking for PIM folders in message store: Master Task List...done
     [11:08:46.957     INFO] Querying MAPI for all data needed to clear flag
     [11:08:46.957     INFO] Data obtained from MAPI. Clearing one at a time
     [11:08:47.174     INFO] Entries cleared: 2. Errors: 0; i: 2

     In the above example we try to clear out all the sync tags in the specified outlook folder. The final line of output shows us that 2 entries had ASynK sync tags, and they have been cleared out.

 2. Delete a profile
     Cardassia:Gout sriramkarra$ python asynk.py --op=del-profile  \
     --name etcbb3 < login.auth
     [11:45:09.153     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-13.11-45-09.153824
     [11:45:09.191     INFO] Logging into Google...
     [11:45:10.994     INFO] Parsing BBDB file /Users/sriramkarra/stuff/code/python/Gout/test/bbdb.olbb...
     Password: [11:45:11.073     INFO] Successfully parsed 2 entries.
     [11:45:11.073     INFO] Fetching contact entries from Google for folder TestBB...
     [11:45:14.310     INFO] Clearing sync state information...
     [11:45:14.313     INFO] Found 0 contacts with matching sync tag(s).
     [11:45:14.321     INFO] Found 0 contacts with matching sync tag(s).
     [11:45:14.322     INFO] Saving BBDB File /Users/sriramkarra/stuff/code/python/Gout/test/bbdb.olbb...
     [11:45:14.655     INFO] Successfully deleted the profile etcbb3 from your Asynk configuration.

     This does two things: remove the sync artifacts from both folders in the profile, and if there are no errors, remove the profile itself from the profile list.

 3. Show a summary of Items in a given folder. Currently implemented for Google Contacts, this the --op=show-folder operation can give you a one line summary of all contacts in google folder.
     Cardassia:Gout sriramkarra$ python asynk.py --op=show-folder --db gc \
     --store karra.etc --folder \
     "http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/2fae451c0fe771d5" \
      < login.auth
     [11:43:01.339     INFO] Debug logging to file: /Users/sriramkarra/stuff/code/python/Gout/logs/asynk_logs.2012-05-13.11-43-01.339125
     [11:43:01.389     INFO] Logging into Google...
     [11:43:03.330     INFO] Contacts.	Name: TestBB;	GID: http://www.google.com/m8/feeds/groups/karra.etc%40gmail.com/base/2fae451c0fe771d5;
     [11:43:03.330     INFO] Summary of contained Items:
     [11:43:06.318     INFO]   Name: Nicholas F. Cianfrocco    Itemid: https://www.google.com/m8/feeds/contacts/karra.etc%40gmail.com/full/86079960ff67a85
     [11:43:06.318     INFO]   Name: K. V. Krishna Kumar       Itemid: https://www.google.com/m8/feeds/contacts/karra.etc%40gmail.com/full/2ed62fa40b90b0bd
     [11:43:06.318     INFO]   Name: ASR Murthy                Itemid: https://www.google.com/m8/feeds/contacts/karra.etc%40gmail.com/full/6c856cd68dae99df
     [11:43:06.318     INFO]   Name: Ed Herring                Itemid: https://www.google.com/m8/feeds/contacts/karra.etc%40gmail.com/full/1041bce089860054
     [11:43:06.318     INFO]   Name: Parveen Kumar Patel       Itemid: https://www.google.com/m8/feeds/contacts/karra.etc%40gmail.com/full/360448d50a2c6bda
     [11:43:06.319     INFO]   Name: Ashwin M. Krishnakumar    Itemid: https://www.google.com/m8/feeds/contacts/karra.etc%40gmail.com/full/10bec2800bd7c9e7

     Here we get the name and Google ID of the contact for all contacts in the specified folder.

Internals
*********

Application and Sync State database: state.json
===============================================

The profile database, and sync state is stored in a file called state.json, which is stored in ~/.asynk/ root directory of the application. This file does not exist in the distribution, but is created the very first time you run ASynK, from a file called state.init.json in the root of the ASynK source directory.

Note for older ASynK users: Before v0.3.0 ASynK could only be executed from the root directory of ASynK source, and the configuration files (state.json and config.json) were read and written there itself. Starting with v0.3.0 ASynK can be invoked from any directory, and it will start using ~/.asynk/ to read and write the configuration files.

If you have been using ASynK before v0.3.0 and have a state.json with your sync profiles and other configurations in the ASynK source directory, you can migrate easily by just running ASynK from its root directory once - ASynK will identify that you have some state and config changes, and will copy your versions to ~/.asynk/ so they are now available for subsequent runs from anywhere. Note, however, tha once copied to ~/.asynk/ only the versions present there will be read and written. Any changes you make to files elsewhere will not be noticed.

Software Architecture
=====================

More to come here

FAQ
***

Here are some Frequently Ask-able Questions, along with answers.

Can I use this to copy my contacts from one Google account to
=============================================================

another? Yes, that is possible.

Is there a way to not use my CardDAV password in clear text?
============================================================

For now it is not possible. But please note that you are running ASynK on your own machine. The authors of ASynK, or any one other than you for that matter, cannot see your activity. That should provide some comfort.

Note that in the early days of ASynK, one had to use password in clear text to sync from/to Google Contacts. Google themselves deprecated this way of authentication, and now it is all oAuth-based. It is a little more cumbersome to get set up in the new way, but it is certainly more secure.

My Google password is not working, but I know it is right!
==========================================================

Circa late 2014 Google starting blocking access to applications accessing account data using plain text authentication as a conservative security policy. You can find more information about this on Google's page here: https://support.google.com/accounts/answer/6010255 and see if this is what's tripping you up.

Other Comparable Tools
======================

What's wrong with Google's tools?
---------------------------------

Google's tools are not as versatile as ASynK. I started work on ASynK because I was unhappy with the state of sync solutions for Android and MS Outlook (more on this later). Subsequently I added BBDB support and the made the application flexible enough that I can sync my contacts between BBDB and MS Outlook directly. Google has no tools to do that, of course. Further, the platform is flexible enough to add support for any number of additional PIM data providers.

Fundamentally, Google does not have a coherent and integrated strategy for Personal Information Management (PIM). Contacts, Calendar, Notes and Tasks are unrelated products with big differences in their APIs. The Tasks program itself is of very limited functionality. This applies equally to Android as well as for the cloud products. So it is hardly surprising that they do not have a tightly integrated PIM sync solution for Android like Blackberry and Nokia's sync software with MS Outlook, say.

Google does provide a few options for people who want to get data into their Google account from Outlook.

 1. A two-way Contacts sync tool for customers of their 'Google Apps for Business users'. That is to say, it is not available for regular non-paying customers like you and me. I haven't tried this even once.

 2. A one-way migration Contacts migration tool from MS Outlook, which only works if you are saying goodbye to Outlook and moving home to Android. This is not practical for people who use Outlook at work, but who want to use their contacts on Android on their phone.

     Even if you're willing to keep all your edits in Outlook, there are various other problems:

       * It is so slow it is impractical for frequent use - migrating 1000 contacts took more than 10 minutes. ASynK takes roughly a tenth of that time.

       * It is very inflexible: all contacts get dumped in your 'My Contacts' mother group. If you would like to neatly isolate your Outlook contacts neatly into a separate group, no go.

       * Older versions of this tool had issues with extracting email addresses from Outlook entries - EX email addresses for those who care. Incidentally, this EX email address problem plagues a number of sync solutions, including an otherwise excellent product from TouchDown for Outlook PST.

 3. A two-way Calendar sync tool that works quite well, and available for all users, include retail consumers.

 4. And finally, their import from CSV. Let's just say, if you're happy with this option you wouldn't be reading this right now.

What is wrong with tools from HTC and other vendors?
----------------------------------------------------

These tools generally sync with MS Outlook, but are not available easily for generic Android users. For e.g. owners of the pure Google phones such as Nexus S. Further these are closed source, and adding BBDB support is, well, not possible.

What is wrong with GoContactSync?
---------------------------------

GoContactSync (http://sourceforge.net/projects/gocontactsync is a GPL-ed two way sync solution between MS Outlook and Google Contacts. It has favourable reviews on the general internet. The drawback is it is written in .NET, a language I do not know and have no desire to learn at this time. Python was versatile enough to work on Windows and Unix, and allowed me to extend to a third PIM data provider (BBDB). As noted earlier, ASynK is the only two way sync solution for BBDB today.

What's Changed between v0.1 and v0.2?
=====================================

It's almost a total rewrite. Here are main changes:

 1. v0.2.0 adds support for Emacs BBDB. At this time ASynK is the only available working program that does bidirectional sync of BBDB to any other PIM database.

 2. v0.1 was designed only for MS Outlook and Google Contacts. As a result it was designed to run on Windows only, and there is a mandatory dependency on PyWin32.

     v0.2 works on Linux, Mac and Windows. Only if MS Outlook support is needed, a third party library called PyWin32 is needed.

 3. v0.1 was distributed as a packaged Windows binary and the only user interface was a GUI built with WxPython (required the user to install an additional third party library)

     v0.2 UI is purely command line, even on Windows.

 4. v0.2 introduces the concept of sync profiles and folder-folder sync, which is much more advanced than what we had earlier. v0.1 always did a sync between the default contacts folder in the default message store with a newly created folder called Gout on google.

GNU AGPL
********

     [file agpl-3.0.texi not included]

GNU FDL
*******

     [file doclicense.texi not included]