This document is applicable for Zimbra Daffodil versions 10.0 and 10.1.0.

License

CC BY-SA Synacor, Inc., 2022-2023

© 2022-2023 by Synacor, Inc. Zimbra Daffodil (v10) Single-Node NG Modules Migration Guide through In-Place Upgrade

This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License unless another license agreement between you and Synacor, Inc. provides otherwise. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0 or send a letter to Creative Commons, PO Box 1866, Mountain View, CA 94042, USA.

Synacor, Inc., 2022-2023
505 Ellicott Street, Suite A39
Buffalo, NY 14203
US

http://www.synacor.com

Licensing changes introduced in Zimbra Daffodil (v10.1)

Zimbra Daffodil (v10.1) onwards, a new licensing system has been introduced which allows more flexibility in managing licenses and enable future granular feature add-ons.

Before you begin with the migration, refer to the Zimbra Daffodil (v10.1) Licensing section and review the changes.

Introduction

This guide provides steps to migrate the 8.8.15/9.0.0 Single-Node NG server to Zimbra Collaboration Daffodil (v10) using the In-place Upgrade method.

You can also refer to this guide for performing migration from your existing 8.8.15-FOSS having Zimbra Suite Plus (ZSP) setup.

Zimbra Collaboration when installed with NG modules installs the following modules:

  • Hierarchical Storage Management.

  • Backup & Restore.

  • Mobile - ActiveSync.

  • Device Management - ABQ.

  • Delegated Admin.

  • Drive.

  • Connect.

The migration is only supported from Zimbra Collaboration 8.8.15 and 9.0.0 versions running the latest patch release.
This migration utility currently does not support the migration of Zimbra Connect / Chat data.
For Backup & Restore feature, Zimbra Collaboration Daffodil (v10) currently does not support storing backup files on S3 external storage or restoring backup from S3 external storage.

When executing these steps, it is recommended to test within a test environment and ensuring there is a backup just in case. Also, this guide is created in steps that can be done once or within different maintenance windows.

Overview

In this method, the server running 8.8.15/9.0.0 server is upgraded to Zimbra Collaboration Daffodil (v10) by running the installer.

If you’re using Zimbra Connect or Chat, please note that migration of these modules is not currently supported. Please contact Zimbra Support or Sales for guidance.

We will provide steps to upgrade the ZCS server only. If you need to upgrade the OS or move to a new system, there are other migration methods and we recommend talking to Zimbra Support for guidance.

This method is suitable for you, if:

  • You want to continue to use the existing Operating System / Hardware.

  • You want to continue to use the existing network/data centre.

Audience

This migration guide assumes you have a thorough understanding of system administration concepts and tasks and are familiar with email communication standards, security concepts, directory services, and database management.

Migration using In-Place Upgrade method

Following are the steps involved in this method:

  1. Verify there is an up-to-date backup and/or take a snapshot before upgrading.

  2. Install the NG migration tool and export NG data.

  3. Prepare the system for upgrade and Document all customization

  4. Upgrade the system.

  5. Import NG data and customization to your new Daffodil (v10) system.

Backup verification

The best practice is to ensure you have a valid backup before upgrading. Also, many environments allow the ability to take a snapshot of your system for quick restoration if needed.

We also recommend exporting your backup or copying your existing backup to a secondary drive or location.

Installing NG Migration tool and Exporting NG Data

In the upgrade to Daffodil (v10) the NG modules will be removed and the new Daffodil (v10) modules will be installed. To prepare for the upgrade, Zimbra created an NG migration tool to export your current NG configuration then after the upgrade of your server, you will then restore the configuration into the Daffodil system.

In many instances, few or none of these steps will need to be run, but it’s all based on how NG modules are being used. Following NG features can be migrated:

  1. NG Core:

    1. HSM

    2. Mobile:

      1. Share Configuration

      2. Device Management - ABQ

    3. Delegated Admin

  2. Drive

If you’re using Zimbra Connect or Chat, please note that migration of these modules is not currently supported. Please contact Zimbra Support or Sales for guidance.
Exporting data does not require downtime but we recommend exporting the data right before the upgrade to ensure it’s the most recent.
The admin user used in export/import command is the Global Admin on the server.

Download and Installing Migration Tool

The migration utility package is available on Zimbra’s public repository.

Following are the steps to download and install the migration utility.

  • On RHEL/CentOS system, execute as root:

 yum clean metadata
 yum check-update
 yum install zimbra-modules-porter

  • On Ubuntu system, execute as root:

 apt-get update
 apt-get install zimbra-modules-porter

After successful installation, a command zmmodulesport will be available on the server.

This utility is version independent and can be installed on an 8.8.15, 9.0.0 or Daffodil (v10) server.

Following are the details on the supported migration modules:

NG Module

Replaced by Zimbra-10 module

Option in utility

Hierarchical Storage Management

Storage Management

hsm

Backup & Restore

Backup & Restore

backup

Mobile

Zimbra Mobile Sync

mobile

ABQ

Zimbra Device Management

mobile

Delegated Admin

Delegated Admin

admin

Drive

Zimbra Briefcase

drive

Export HSM

Customers not using HSM OR using Non-S3 volumes can skip this step.

To export HSM data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=hsm --filename=hsm-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/hsm-export.log

A file hsm-data.tgz will be created within /opt/zimbra/tmp/modules/hsm/.

Export Delegated Admin

To export Delegated Admin data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=admin --filename=admin-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/da-export.log

This will create a file admin-data.tgz under /opt/zimbra/tmp/modules/admin.

Once the system has been upgraded to Daffodil, Delegated Admins will not be able to manage accounts until NG Delegated Admin data is imported or Daffodil admin rights are enabled.

Export Mobile Configuration

Customers who are using ABQ and/or ActiveSync shared folder will need to migrate their configuration. Mobile migration data consists of ABQ rules and share folder configuration for mobile devices.

To export Mobile data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=mobile --filename=mobile-data1 --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/mobile-export.log

This will create a file mobile-data1.tgz under /opt/zimbra/tmp/modules/mobile.

Export Backup & Restore Configuration

The migration utility only provides the information if the Backup & Restore feature is enabled/disabled for the COS.

To export Backup data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport system export --modules=backup --filename=backup-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/br-export.log

This will create a file backup-data.tgz under /opt/zimbra/tmp/modules/backup.

Import Backup & Restore Configuration
The Backup & Restore configuration cannot be imported through the utility. You will have to manually update the configuration for the domains/users on your system.
Daffodil (v10) and NG backups contain different format and you will not be able to directly use the Daffodil (v10) Backup & Restore feature to restore your NG backups. Also, if the NG backup is used for anything other than disaster recovery, we recommend copying the NG backup to an archive location. Zimbra has created a tool to allow the restoration of message data for auditing and recovery purposes only.

To manually review the backup configuration:

  • Extract the exported backup-data.tgz file.

  • In the extracted directory, open the json file and check the COS having Enabled value.

  • Log into server.com’s Admin Console.

  • Go to Configure → Global Settings → Backup/Restore.

  • Under Select users for backup → Domain and COS, click on Add COS button and add COS for which backup was enabled in the json file.

  • Repeat the steps if there are multiple COS. Click on Save.

Once the backup configuration has been set, a full backup will be created for any new account that was migrated each night. Backups can manually execute once the targeted accounts are migrated for the day.

Export Drive Data

Customers using Drive will need to export the user’s drive data so it can be imported into the user Briefcase after the upgrade.

To export Drive data using zmmodulesport, execute the command as zimbra user:

su - zimbra
zmmodulesport account export --modules=drive --filename=drive-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/drive-export.log

This will create a file drive-data.tgz under /opt/zimbra/tmp/modules/drive.

Drive data can be exported for a single user, multiple users, or all accounts on the system, the following are the options that can use when creating the request:

The account option supports following:

Option

Description

--accounts=*

Export all accounts.

--accounts=name@domain

Export a single account.

--accounts=name1@domain1,name2@domain2

Export multiple accounts

--accounts=/path/to/accounts-file.txt

Export accounts listed in the file. The accounts have to be listed on new lines.

--accounts=domain

Export all available accounts for the specified domain on the server.
It is recommended to use --accounts=* option for the systems where the user’s Drive data is not huge in size.

Disable Mobile Sync before Upgrade

Since ABQ rules have not been imported at this point and will be done after the upgrade, it is recommended to disable Mobile Sync for all the users. This is to restrict the configured devices from syncing immediately after the upgrade. It may also cause spike in CPU and IO due to the amount of data being synced.

After the upgrade, we will import the ABQ rules and then enable Mobile Sync which will enforce connected device to honour the ABQ rules.

zmprov ma <account@domain.com> zimbraFeatureMobileSyncEnabled FALSE

Customization and Preparing your upgrade.

In-place upgrades will remove most customizations. Before starting the upgrade, we recommend reviewing your system and documenting all customizations so that once the upgrade has been completed, you can verify and reapply any missing customization if needed.

Upgrading to Daffodil

It’s time to upgrade your system. Download and uncompress the Daffodil installer which can be obtained from the download page. We also recommend closing all ports to the mailbox server to prevent new messages, devices, and users from accessing the system during the upgrade.

Disable the NG modules by running:

zmprov ms `zmhostname` zimbraNetworkModulesNGEnabled FALSE

Remove NG Modules:

zmzimletctl undeploy com_zextras_client
zmzimletctl undeploy com_zextras_drive_open
zmzimletctl undeploy com_zextras_zextras
zmzimletctl undeploy com_zextras_drive

Add the OnlyOffice repository on the system.

If you are not using Zimbra Docs or do not intend to use it after the upgrade to Daffodil (v10), then you can skip this section.

Following are the steps. Execute the commands as root user

  • RHEL 7

$ cat > /etc/yum.repos.d/zimbra-onlyoffice.repo <<EOF
[zimbra-onlyoffice]
name=Zimbra Onlyoffice RPM Repository
baseurl=https://repo.zimbra.com/rpm/onlyoffice/rhel7
gpgcheck=1
enabled=1
EOF

$ yum --disablerepo=* --enablerepo=zimbra-onlyoffice clean metadata

$ yum check-update --disablerepo=* --enablerepo=zimbra-onlyoffice --noplugins

  • RHEL 8

$ cat > /etc/yum.repos.d/zimbra-onlyoffice.repo <<EOF
[zimbra-onlyoffice]
name=Zimbra Onlyoffice RPM Repository
baseurl=https://repo.zimbra.com/rpm/onlyoffice/rhel8
gpgcheck=1
enabled=1
EOF

$ yum --disablerepo=* --enablerepo=zimbra-onlyoffice clean metadata

$ yum check-update --disablerepo=* --enablerepo=zimbra-onlyoffice --noplugins

  • UBUNTU 18

$ cat > /etc/apt/sources.list.d/zimbra-onlyoffice.list << EOF
deb [arch=amd64] https://repo.zimbra.com/apt/onlyoffice bionic zimbra
deb-src [arch=amd64] https://repo.zimbra.com/apt/onlyoffice bionic zimbra
EOF

$ apt-get update

  • UBUNTU 20

$ cat > /etc/apt/sources.list.d/zimbra-onlyoffice.list << EOF
deb [arch=amd64] https://repo.zimbra.com/apt/onlyoffice focal zimbra
deb-src [arch=amd64] https://repo.zimbra.com/apt/onlyoffice focal zimbra
EOF

$ apt-get update

Upgrade System

Run the install.sh with the -skip-ng-check flag and the first question will be about the license agreement:

Do you agree with the terms of the software license agreement? [N] Y

If you received the following warning, it means the skip-ng-check flag wasn’t used:

NG Modules detected on this system. If you continue with this upgrade, NG module packages and the associated data will be deleted. If you want to preserve NG data, consider migrating or a rolling upgrade strategy for upgrading your system. For more information, please contact Zimbra Support. If you still want to continue, start upgrade using --skip-ng-check

After accepting the license, the Daffodil installer will inform you that a backup does not exist and asks if you wish to continue.

Do you wish to continue without a backup? [N] Y

This occurs because it is checking to see if there is a Daffodil backup and at this time one hasn’t been created. Since there is an NG backup and or a snapshot, select Y and proceed.

The installer will ask if you like to do a message store database integrity check, select enter to proceed with the check:

Do you want to verify message store database integrity? [Y]

You will be asked if you wish to use the zimbra repository and if you wish to proceed to upgrade, select enter to proceed.

Use Zimbra's package repository [Y]
Do you wish to upgrade? [Y]

The installer will ask if you like to install packages that have not been installed, two new packages with Daffodil are zimbra-onlyoffice and chat and video. zimbra-onlyoffice is available for all licensed customers and can be installed at this time or at a later date. Customers who want to use Zimbra’s Document and Collaboration feature, zimbra-onlyoffice should be installed at this time.

For customers currently using Zimbra chat and video, please contact Zimbra Sales or open a support case for more information about chat and video within Daffadil. For customers who are interested in the new chat and video demo, please contact Zimbra Sales for more information. 
Install zimbra-archiving [N]
Install zimbra-onlyoffice [N] Y
    Upgrading zimbra-license-tools
    Upgrading zimbra-license-extension
    Upgrading zimbra-network-store
    Upgrading zimbra-patch
    Upgrading zimbra-mta-patch
    Upgrading zimbra-proxy-patch
Install chat and video features [N] Y

Once you selected the new packages, the installer will ask if you want to upgrade:

The system will be modified.  Continue? [N]

Select Y then enter to proceed with the upgrade. Once Completed you will see the following:

Finished installing network zimlets.
Getting list of all zimlets...done.
Updating non-standard zimlets...
Finished updating non-standard zimlets.
Restarting mailboxd...done.
Skipping creation of default domain GAL sync account - existing install detected.
Setting up zimbra crontab...done.

Moving /tmp/zmsetup.yyyymmdd-hhmmss.log to /opt/zimbra/log

Configuration complete - press return to exit

Your system has been upgraded to Daffodil.

Remove Zimbra Docs

If you are running Zimbra Docs on a separate node, you can remove the server details from the Daffodil (v10) LDAP tree after the upgrade.

Execute this command as a zimbra user:

su - zimbra
zmprov ds docs.server.com

Import NG Data and apply customizations

The final stage of upgrade consists of importing the NG data and applying any customizations you have on your server.

Import HSM

You can skip this section if are not using External Store providers.

It is critical to restore HSM configuration before making the server live or doing any testing. This will ensure that the emails are accessible after the import.

In Daffodil (v10), the locator format for blobs in S3 volume is different than the NG HSM. The locator is the pointer to the blob location in the database for each message. Thus, while importing the HSM configuration, additional options has to be included in the command to update the locator for all the user’s on the server.

Following are the details on the additional required options:

  • -t=inplace - Option to be included for an In-place upgrade method.

  • -l - Option to enable the update of the locators.

  • -uam - Option to update the locator for all the mailboxes. Set it to TRUE when importing the HSM data so that the locators are updated to the new format while importing the HSM data.

Please refer to admin-guide for more details.

In case the locator option is not passed to the import command, the locator’s will not be updated to the new format and user’s might see Missing Blob error when viewing the email.

  • As a zimbra user, execute this command:

su - zimbra
zmmodulesport system import --modules=hsm --filename=hsm-data -t=inplace --server=server.com --username=admin@domain.com --password=admin_passwd -l=true -uam=true >> /opt/zimbra/log/hsm-import.log

In case the locator’s are not updated during the import, they can be updated after the HSM data has been migrated. As a zimbra user, execute this command:

su - zimbra
zmmodulesport system locatorUpgrade --modules=hsm --filename=hsm-data -uam=true --username=admin@domain.com --password=admin_passwd

Import Delegated Admin

  • As a zimbra user, execute this command:

su - zimbra
zmmodulesport system import --modules=admin --filename=admin-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/da-import.log

Import Mobile Data

For NG Mobile, if you have set the ABQ mode to Strict and if there are any devices updated in the Allowed Device list on the NG server, after the mobile data import to the server, update the zimbraMobileBlockedDevices to allow syncing of the allowed devices:

  • As a zimbra user, execute this command:

su - zimbra
zmprov md new.server.com +zimbraMobileBlockedDevices "*"

  • As a zimbra user, execute this command:

su - zimbra
zmmodulesport system import --modules=mobile --filename=mobile-data1 --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/mobile-import.log
After the import, user’s who are using NG Mobile Password feature will have to update their zimbra account’s password in mobile settings.

Import Drive Data

Before you start the import, it is recommended to increase the zimbraAdminAuthTokenLifetime so in case of large data transfer, the session is not timed out. The default value of zimbraAdminAuthTokenLifetime is 12 hour (12h).

On COS level, if you want to increase zimbraAdminAuthTokenLifetime value to 24 hours, please specify the value as 24h and update the zimbraAdminAuthTokenLifetime attribute on mailbox server.

su - zimbra
zmprov mc default zimbraAdminAuthTokenLifetime 24h

  • As a zimbra user, restart mailbox service to make the changes effective:

zmmailboxdctl stop

  • To import the Drive data, execute this command as a zimbra user:

su - zimbra
zmmodulesport account import --modules=drive --filename=drive-data --server=server.com --username=admin@domain.com --password=admin_passwd >> /opt/zimbra/log/drive-import.log

Once your configuration has been added and tested, you can enable mail flow and access to the user accounts.

Enabling Daffodil Backup

Log into your Global admin account, select Configuration → Global Setting → Backup/Restore and enable backup. Review the backup configuration to ensure it meets your needs.

Device Re-sync after Upgrade

Once the server has been upgraded, you can stagger the enabling of mobile on accounts to prevent all accounts from re-syncing at the same time.

zmprov ma <account@domain.com> zimbraFeatureMobileSyncEnabled TRUE
This is just a recommendation and each environment is different. It is up to the system Admin to choose the best method that’s supported by their environment.