Couchbase - Clone Records and Update Document Key

Moving selected documents from one Couchbase bucket to another is straightforward when the document keys remain unchanged. The more difficult case is when the migration also requires a different key for each document.

In the Couchbase version used for this article, the document key could not be updated directly through N1QL. The workaround below stages the documents in a temporary bucket, modifies the backup representation, restores the data, and then moves the documents to the final bucket.

Version context: This is a historical workflow based on the Couchbase backup tooling and file formats available when the article was written. Current Couchbase versions provide different backup, restore, and migration capabilities. Treat this as a documented solution for that environment rather than a recommendation for modern deployments.

Step 1: Copy the Target Documents to a Temporary Bucket

First, select the documents that need to be migrated and copy them into a temporary bucket.

UPSERT INTO `bucket-tmp` (KEY _k, VALUE _v) SELECT META().id _k, _v FROM `bucket-1` _v WHERE _v.some_field="some_value"....

The original bucket-to-bucket migration query is shown here for reference:

UPSERT INTO `bucket-2` (KEY _k, VALUE _v) SELECT META().id _k, _v FROM `bucket-1` _v WHERE _v.some_field="some_value"....

Step 2: Back Up the Temporary Bucket

Use cbbackup to create a backup of the temporary bucket.

sh cbbackup -m full http://localhost:8091 /path/tmp_backup -u username -p password -b bucket-tmp -x data_only=1

In clustered environments, the backup can contain multiple .cbb files. Apply the following steps to each relevant file.

Step 3: Open the Backup Database

Open the .cbb file with DB Browser for SQLite. The backup contains tables including cbb_meta and cbb_msg; the document records are stored in cbb_msg.

Before changing anything, keep an untouched copy of the backup so the original can be restored if required.

Step 4: Update the Document Keys

Use a SQL statement to transform the key values. The exact expression depends on how the destination keys should be generated. The following example replaces part of an existing key:

update cbb_msg set key = replace(key, 'abc', 'xyz')

Validate the affected rows before saving the change:

select key from cbb_msg

After validating the result, use Write Changes in DB Browser for SQLite. Repeat the process for each .cbb file when necessary.

Step 5: Restore the Modified Backup

Flush or recreate the temporary bucket as appropriate for the test environment, then restore the modified backup using cbrestore.

sh cbrestore /path/tmp_backup http://localhost:8091 -u username -p password -B bucket-tmp -x rehash=1

Step 6: Move the Documents to the Destination Bucket

Once the temporary bucket contains the documents under their new keys, migrate them to the final bucket.

UPSERT INTO `bucket-2` (KEY _k, VALUE _v) SELECT META().id _k, _v FROM `bucket-tmp` _v

Operational Considerations

This workaround modifies backup data outside the normal Couchbase data-access path, so it should be treated cautiously. Validate the process on non-production data first, retain backups, verify document counts and keys before and after restore, and confirm that application references to the old keys are handled appropriately.

Takeaway

The underlying lesson is that a document key is part of a document's identity, not simply another mutable field. When a migration requires changing that identity, the migration plan needs an explicit strategy for recreating or restoring documents under new keys rather than treating the operation as a normal field update.