Zimbra Daffodil (v10) Multi-Node NG Modules Migration Guide through In-Place Upgrade

This document is applicable for Zimbra Daffodil versions 10.0 and 10.1.0.

License

CC BY-SA Synacor, Inc., 2022-2023

This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License unless another license agreement between you and Synacor, Inc. provides otherwise. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0 or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

Synacor, Inc., 2022-2023
505 Ellicott Street, Suite A39
Buffalo, NY 14203
US

http://www.synacor.com

Licensing changes introduced in Zimbra Daffodil (v10.1)

Zimbra Daffodil (v10.1) onwards, a new licensing system has been introduced which allows more flexibility in managing licenses and enable future granular feature add-ons.

Before you begin with the migration, refer to the Zimbra Daffodil (v10.1) Licensing section and review the changes.

Introduction

This guide provides steps to migrate the 8.8.15/9.0.0 multi-server environment using the in-place migration method to Zimbra Collaboration Daffodil (v10). The in-place migration method allows you to install the Daffodil version onto the current production systems. To be able to use this method, the operating system that is being used must be supported by your current production version and Daffodil.

You can also refer to this guide for performing migration from your existing 8.8.15-FOSS having Zimbra Suite Plus (ZSP) setup.

The in-place upgrade also supports instances with or without NG modules. Zimbra Collaboration when installed with NG modules installs the following modules:

Hierarchical Storage Management.
Backup & Restore.
Mobile - ActiveSync.
Device Management - ABQ.
Delegated Admin.
Drive.
Connect.

The migration is only supported from Zimbra Collaboration 8.8.15 and 9.0.0 versions running the latest patch release.

This migration utility currently does not support the migration of Zimbra Connect / Chat data.

For Backup & Restore feature, Zimbra Collaboration Daffodil (v10) currently does not support storing backup files on S3 external storage or restoring backup from S3 external storage.

When executing these steps, it is recommended to test within a test environment and ensuring there is a backup just in case. Also, this guide is created in steps that can be done once or within different maintenance windows.

Audience

This migration guide assumes you have a thorough understanding of system administration concepts and tasks and are familiar with email communication standards, security concepts, directory services, and database management.

Overview and Pre-requisites

A multi-server environment is an environment where one or more Zimbra components are installed on separate servers. The environment may vary from customer to customer, but a common environment is where multi-instances of each module (LDAP, proxy, MTA, and Mailbox) are installed on an independent server.

To perform an in-place upgrade, the current operating system must support the Daffodil (v10) server.

When upgrading a multi-server environment, the upgrade process is to upgrade all modules within each component in the following order:

LDAP server
Proxy server
MTA server
Mailbox servers

For environments that are running more than one of the non-mailbox modules on the same server, you will need to upgrade the servers containing the LDAP first, then upgrade the remaining non-mailbox server before starting the upgrade of any mailbox servers.

For customers running NG modules, NG modules are installed on the Mailbox servers and there are no changes to the upgrade process for non-mailbox modules.

In the upgrade process, Zimbra recommends the following steps:

Test and verify the upgrade process.
Verify there is an up-to-date backup and/or take a snapshot before upgrading.
Prepare the system for upgrade and document all customization.
Perform in-place upgrade of LDAP, Proxy, and then MTA server/s.
On Mailbox server/s, install the NG migration tool and export NG data.
Stop all traffic to the Mailbox server’s.
Perform an in-place upgrade on the Mailbox server.
Import NG data and customization to your new Daffodil (v10) system.
Test the upgrade system.
Enable traffic

The following is a high level of the upgrade process of the non-mailbox servers, but if you need more guidance than what is provided below, please open a support case and Zimbra Support can assist you:

Identify, document, and remove any customization to the LDAP, Proxy, MTA
Download and uncompress the Daffodil (v10) installer which can be obtained from the download page.
Run the install.sh and follow the instructions on the screen to upgrade the nodes.
Reinstall customization

At this time all of the non-mailbox servers are running Daffodil (v10) and all mailbox servers are running the pre-upgrade version.

Installing NG Migration tool on Mailbox server.

In the upgrade to Daffodil (v10) the NG modules will be removed and the new Daffodil (v10) modules will be installed. To prepare for the upgrade, Zimbra created an NG migration tool to export your current NG configuration then after the upgrade of your server, you will then restore the configuration into the Daffodil (v10) system.

In many instances, few or none of these steps will need to be run, but it’s all based on how NG modules are being used. Following NG features can be migrated:

NG Core:
1. HSM
2. Mobile:
  1. Share Configuration
  2. Device Management - ABQ
3. Delegated Admin
Drive

Other migration features available within the migration tool like Hierarchical Storage Management for external storage and backup are not needed or are not supported.

If you’re using Zimbra Connect or Chat, please note that migration of these modules is not currently supported. Please contact Zimbra Support or Sales for guidance.

Exporting data does not require downtime but we recommend exporting the data right before the upgrade to ensure it’s the most recent.

The admin user used in export/import command is the Global Admin on the server.

Download and Installing Migration Tool

The migration utility package is available on Zimbra’s public repository and will need to be installed on each Daffodil (v10) mailbox server.

Following are the steps to download and install the migration utility.

On RHEL/CentOS system, execute as root:

 yum clean metadata
 yum check-update
 yum install zimbra-modules-porter

On Ubuntu system, execute as root:

 apt-get update
 apt-get install zimbra-modules-porter

After successful installation, a command zmmodulesport will be available on the server.

There are 3 types of data that can be migrated:

Global Data - Needs to be exported/imported only once before/after upgrading any mailbox server.
Server Data - Needs to be exported/imported before/after upgrading each mailbox server.
User Data - Needs to be exported/imported for the users belonging to the mailbox server before/after upgrading each mailbox server.

Following are the details on the supported migration modules:

NG Module

Replaced by Zimbra-10 module

Option in utility

Type

Hierarchical Storage Management

Storage Management

hsm

Server

Backup & Restore

backup

Server

Mobile

Zimbra Mobile Sync

mobile

Global

ABQ

Zimbra Device Management

mobile

Global

Delegated Admin

admin

Global

Drive

Zimbra Briefcase

drive

User

Exporting NG Data on Mailbox server

Export HSM

Customers not using HSM OR using Non-S3 volumes can skip this step.

Since it’s a Server data, you need to export it for each mailbox server.

To export HSM data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=hsm --filename=hsm-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/hsm-export.log

A file hsm-data.tgz will be created within /opt/zimbra/tmp/modules/hsm/.

Export Delegated Admin

Since it’s a Global data, you need to export it only once from the first mailbox server you are going to upgrade.

To export Delegated Admin data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=admin --filename=admin-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/da-export.log

This will create a file admin-data.tgz under /opt/zimbra/tmp/modules/admin.

Once the system has been upgraded to Daffodil, Delegated Admins will not be able to manage accounts until NG Delegated Admin data is imported or Daffodil admin rights are enabled.

Export Mobile Configuration

Since it’s a Global data, you need to export it only once from the first mailbox server you are going to upgrade.

Customers who are using ABQ and/or share folder sync will need to migrate their configuration. Mobile migration data consists of ABQ rules and share folder configuration for mobile devices.

To export Mobile data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=mobile --filename=mobile-data1 --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/mobile-export.log

This will create a file mobile-data1.tgz under /opt/zimbra/tmp/modules/mobile.

Export Backup & Restore Configuration

Since it’s a Server data, you need to export it for each mailbox server.

The migration utility only provides the information if the Backup & Restore feature is enabled/disabled for the COS.

To export Backup data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=backup --filename=backup-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/br-export.log

This will create a file backup-data.tgz under /opt/zimbra/tmp/modules/backup.

Import Backup & Restore Configuration

Since it’s a Server data, you need to import it for each mailbox server.

The Backup & Restore configuration cannot be imported through the utility. You will have to manually update the configuration for the domains/users on your system.

Daffodil (v10) and NG backups contain different format and you will not be able to directly use the Daffodil (v10) Backup & Restore feature to restore your NG backups. Also, if the NG backup is used for anything other than disaster recovery, we recommend copying the NG backup to an archive location. Zimbra has created a tool to allow the restoration of message data for auditing and recovery purposes only.

To manually review the backup configuration:

Extract the exported backup-data.tgz file.
In the extracted directory, open the json file and check the COS having Enabled value.
Log into server.com’s Admin Console.
Go to Configure → Global Settings → Backup/Restore.
Under Select users for backup → Domain and COS, click on Add COS button and add COS for which backup was enabled in the json file.
Repeat the steps if there are multiple COS. Click on Save.

Once the backup configuration has been set, a full backup will be created for any new account that was migrated each night. Backups can manually execute once the targeted accounts are migrated for the day.

Export Drive Data

Since it’s a User data, you need to export it for each mailbox server which will export the data for the accounts belonging to that mailbox server.

Customers using Drive will need to export the user’s drive data so it can be imported into the user Briefcase after the upgrade.

To export Drive data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport account export --modules=drive --filename=drive-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/drive-export.log

This will create a file drive-data.tgz under /opt/zimbra/tmp/modules/drive.

Drive data can be exported for a single user, multiple users, or all accounts on the system, the following are the options that can use when creating the request:

The account option supports following:

Option

Description

--accounts=*

Export all accounts.

--accounts=name@domain

Export a single account.

--accounts=name1@domain1,name2@domain2

Export multiple accounts

--accounts=/path/to/accounts-file.txt

Export accounts listed in the file. The accounts have to be listed on new lines.

--accounts=domain

Export all available accounts for the specified domain on the server.

It is recommended to use --accounts=* option for the systems where the user’s Drive data is not huge in size.

Disable Mobile Sync before Upgrade

Since ABQ rules have not been imported at this point and will be done after the upgrade, it is recommended to disable Mobile Sync for all the users. This is to restrict the configured devices from syncing immediately after the upgrade. It may also cause spike in CPU and IO due to the amount of data being synced.

After the upgrade, we will import the ABQ rules and then enable Mobile Sync which will enforce connected device to honour the ABQ rules.

zmprov ma <account@domain.com> zimbraFeatureMobileSyncEnabled FALSE

Customization and Preparing your upgrade.

In-place upgrades will remove most customizations. Before starting the upgrade, we recommend reviewing your system and documenting all customizations so that once the upgrade has been completed, you can verify and reapply any missing customization if needed.

Upgrade Mailbox server to Daffodil (v10)

It’s time to upgrade your mailbox server. Download and uncompress the Daffodil (v10) installer which can be obtained from the download page. We also recommend closing all ports to the mailbox server to prevent new messages, devices, and users from accessing the system during the upgrade.

Disable the NG modules by running:

zmprov ms `zmhostname` zimbraNetworkModulesNGEnabled FALSE

Remove NG Modules:

zmzimletctl undeploy com_zextras_client
zmzimletctl undeploy com_zextras_drive_open
zmzimletctl undeploy com_zextras_zextras
zmzimletctl undeploy com_zextras_drive

Add the OnlyOffice repository on the system.

If you are not using Zimbra Docs or do not intend to use it after the upgrade to Daffodil (v10), then you can skip this section.

Following are the steps. Execute the commands as root user

RHEL 7

$ cat > /etc/yum.repos.d/zimbra-onlyoffice.repo <<EOF
[zimbra-onlyoffice]
name=Zimbra Onlyoffice RPM Repository
baseurl=https://repo.zimbra.com/rpm/onlyoffice/rhel7
gpgcheck=1
enabled=1
EOF

$ yum --disablerepo=* --enablerepo=zimbra-onlyoffice clean metadata

$ yum check-update --disablerepo=* --enablerepo=zimbra-onlyoffice --noplugins

RHEL 8

$ cat > /etc/yum.repos.d/zimbra-onlyoffice.repo <<EOF
[zimbra-onlyoffice]
name=Zimbra Onlyoffice RPM Repository
baseurl=https://repo.zimbra.com/rpm/onlyoffice/rhel8
gpgcheck=1
enabled=1
EOF

$ yum --disablerepo=* --enablerepo=zimbra-onlyoffice clean metadata

$ yum check-update --disablerepo=* --enablerepo=zimbra-onlyoffice --noplugins

UBUNTU 18

$ cat > /etc/apt/sources.list.d/zimbra-onlyoffice.list << EOF
deb [arch=amd64] https://repo.zimbra.com/apt/onlyoffice bionic zimbra
deb-src [arch=amd64] https://repo.zimbra.com/apt/onlyoffice bionic zimbra
EOF

$ apt-get update

UBUNTU 20

$ cat > /etc/apt/sources.list.d/zimbra-onlyoffice.list << EOF
deb [arch=amd64] https://repo.zimbra.com/apt/onlyoffice focal zimbra
deb-src [arch=amd64] https://repo.zimbra.com/apt/onlyoffice focal zimbra
EOF

$ apt-get update

Upgrade System

Run the install.sh with the -skip-ng-check flag and the first question will be about the license agreement:

Do you agree with the terms of the software license agreement? [N] Y

If you received the following warning, it means the skip-ng-check flag wasn’t used:

NG Modules detected on this system. If you continue with this upgrade, NG module packages and the associated data will be deleted. If you want to preserve NG data, consider migrating or a rolling upgrade strategy for upgrading your system. For more information, please contact Zimbra Support. If you still want to continue, start upgrade using --skip-ng-check

After accepting the license, the Daffodil installer will inform you that a backup does not exist and asks if you wish to continue.

Do you wish to continue without a backup? [N] Y

This occurs because it is checking to see if there is a Daffodil backup and at this time one hasn’t been created. Since there is an NG backup and or a snapshot, select Y and proceed.

The installer will ask if you like to do a message store database integrity check, select enter to proceed with the check:

Do you want to verify message store database integrity? [Y]

You will be asked if you wish to use the zimbra repository and if you wish to proceed to upgrade, select enter to proceed.

Use Zimbra's package repository [Y]
Do you wish to upgrade? [Y]

The installer will ask if you like to install packages that have not been installed, two new packages with Daffodil are zimbra-onlyoffice and chat and video. zimbra-onlyoffice is available for all licensed customers and can be installed at this time or at a later date. Customers who want to use Zimbra’s Document and Collaboration feature, zimbra-onlyoffice should be installed at this time.

For customers currently using Zimbra chat and video, please contact Zimbra Sales or open a support case for more information about chat and video within Daffadil. For customers who are interested in the new chat and video demo, please contact Zimbra Sales for more information.

Install zimbra-archiving [N]
Install zimbra-onlyoffice [N] Y
    Upgrading zimbra-license-tools
    Upgrading zimbra-license-extension
    Upgrading zimbra-network-store
    Upgrading zimbra-patch
    Upgrading zimbra-mta-patch
    Upgrading zimbra-proxy-patch
Install chat and video features [N] Y

Once you selected the new packages, the installer will ask if you want to upgrade:

The system will be modified.  Continue? [N]

Select Y then enter to proceed with the upgrade. Once Completed you will see the following:

Finished installing network zimlets.
Getting list of all zimlets...done.
Updating non-standard zimlets...
Finished updating non-standard zimlets.
Restarting mailboxd...done.
Skipping creation of default domain GAL sync account - existing install detected.
Setting up zimbra crontab...done.

Moving /tmp/zmsetup.yyyymmdd-hhmmss.log to /opt/zimbra/log

Configuration complete - press return to exit

Your system has been upgraded to Daffodil.

Import NG Data and apply customizations

The final stage of upgrade consists of importing the NG data and applying any customizations you have on your server.

Import HSM

Since it’s a Server data, you need to import it for each mailbox server.

It is critical to restore HSM configuration before making the server live or doing any testing. This will ensure that the emails are accessible after the import. This is a server configuration and the configuration will need to be obtained from and restored to each store.

In Daffodil (v10), the locator format is different than the NG HSM. The locator is the pointer to the blob location in the database for each message. Thus, while importing the HSM configuration, an additional option has to be included in the command to update the locator for all the user’s on the server.

Following are the details on the options:

--t=inplace - Option to be included for In-place upgrade method.
-l - Option to enable the update of the locators.
-uam - Option to update the locator for all mailboxes. Set it to TRUE when importing the HSM data so that the locators are updated to the new format while importing the HSM data.

Please refer to admin-guide for more details.

In case the locator option is not passed to the import command, the locator’s will not be updated to the new format and user’s might see Missing Blob error when viewing the email.

As a zimbra user, execute this command:

su - zimbra
zmmodulesport system import --modules=hsm --filename=hsm-data --t=inplace --server=server.com --username=admin@domain.com --password=admin_passwd -l=true -uam=true >> /opt/zimbra/log/hsm-import.log

In case the locator’s are not updated during the import, they can be updated after the HSM data has been migrated. As a zimbra user, execute this command:

su - zimbra
zmmodulesport system locatorUpgrade --modules=hsm --filename=hsm-data -uam=true --username=admin@domain.com --password=admin_passwd

Import Delegated Admin

Since it’s a Global data, you need to import it only once from the first mailbox server you are going to upgrade.

As a zimbra user, execute this command:

su - zimbra
zmmodulesport system import --modules=admin --filename=admin-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/da-export.log

Import Mobile Data

Since it’s a Global data, you need to import it only once from the first mailbox server you are going to upgrade.

As a zimbra user, execute this command:

su - zimbra
zmmodulesport system import --modules=mobile --filename=mobile-data1 --server=server.com --username=admin@domain.comm --password=admin_passwd >> /opt/zimbra/log/mobile-import.log

After the import, user’s who are using NG Mobile Password feature will have to update their zimbra account’s password in mobile settings.

For NG Mobile, if you have set the ABQ mode to Strict and if there are any devices updated in the Allowed Device list on the NG server, after the mobile data import to the server, update the zimbraMobileBlockedDevices to allow syncing of the allowed devices:

As a zimbra user, execute this command:

su - zimbra
zmprov md domain.com +zimbraMobileBlockedDevices "*"

Import Drive Data

Since it’s a User data, you need to import it for each mailbox server which will export the data for the accounts belonging to that mailbox server.

Before you start the import, it is recommended to increase the zimbraAdminAuthTokenLifetime so in case of large data transfer, the session is not timed out. The default value of zimbraAdminAuthTokenLifetime is 12 hour (12h).

On COS level, if you want to increase zimbraAdminAuthTokenLifetime value to 24 hours, please specify the value as 24h and update the zimbraAdminAuthTokenLifetime attribute on mailbox server.

su - zimbra
zmprov mc default zimbraAdminAuthTokenLifetime 24h

Restart mailbox service to make the changes effective:

su - zimbra
zmmailboxdctl stop

To import the Drive data, execute this command as a zimbra user:

su - zimbra
zmmodulesport account import --modules=drive --filename=drive-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/drive-import.log

Once your configuration has been added and tested, you can enable mail flow and access to the user accounts.

Enabling Daffodil Backup

Log into your Global admin account, select Configuration → Global Setting → Backup/Restore and enable backup. Review the backup configuration to ensure it meets your needs.

Device Re-sync after Upgrade

Once the server has been upgraded, you can stagger the enabling of mobile on accounts to prevent all accounts from re-syncing at the same time.

zmprov ma <account@domain.com> zimbraFeatureMobileSyncEnabled TRUE

This is just a recommendation and each environment is different. It is up to the system Admin to choose the best method that’s supported by their environment.