How do I keep track of progress of a Cloudant replication? - database

Question

If I have an IBM Cloudant database with around one million documents in it and I set up a replication process to copy this data to another region, how can I tell how far the replication job has progressed? I know when it starts and when it has finished, but nothing in between. Is there a way to track progress?

Answer 1

If the target database was empty at the start of the replication, and there's no other changes being written to the target, then it's a case of waiting until the sequence token of the target database matches the sequence token of the source.
You can find the current sequence token of a database by using the GET /<database name> endpoint on the source & the target e.g.
curl $URL/sourcedb
{
"update_seq": "23600-g1AAAARXeJyd0",
"db_name": "sourcedb",
...
}
In the above example, there are 23600 changes that the replicator needs to work through. The same command can be run against the target database to see the progress of replication.
Alternatively, there is an API endpoint that allows you to view the replication job's progress: GET _scheduler/docs/_replicator/<replication id> where the replication id is the _id of the document in the _replicator database that was created to initiate the replication.
It returns an object like this:
{
"database": "_replicator",
"doc_id": "e0330b1936f6194da22af8fa663c5be8",
"id": null,
"source": "https://source.cloudant.com/sourcedb/",
"target": "https://target.cloudant.com/targetdb/",
"state": "completed",
"error_count": 0,
"info": {
"revisions_checked": 1005,
"missing_revisions_found": 1005,
"docs_read": 1005,
"docs_written": 1005,
"changes_pending": 376,
"doc_write_failures": 0,
"checkpointed_source_seq": "1011-g1AAAAfLeJy91FFKwzAYwPGigo_uBvqq0JmkbZqCsomoj3oDzZcvZYxtFbc96w30BnoDvYHeQG-gN9AbzCYpbntbhfQlhdJ"
},
"start_time": "2020-11-17T09:55:01Z",
"last_updated": "2020-11-17T09:55:58Z"
}
which includes the status of the replication, how many documents have been processed and the last checkpointed sequence token, which should be enough to estimate progress of the replication job.
The full details of the API call are here.