Zimbra Daffodil (v10) Multi-Node NG Modules Migration Guide through Rolling-Upgrade

Environments running a single master LDAP with or without slave/replicas will need to enable MMR using the following steps, if MMR is enabled, this section can be skipped.

Before starting this process, new systems using an OS version that is supported by the current production version Daffodil (v10) and Daffodil needs to be created. This system should have the required environmental configuration (CPU, Memory, etc) to support current and future growth. Operating system (OS) are server-specific and does not effect application level functionality and OS provider and version can differ on each server.

Before enabling MMR, you will need to obtain existing passwords from #.old.ldap.com to use during the installation on #.new.ldap.com. Please save these passwords as we will need them when configuring LDAP MMR setup.

A common issue is the LDAP port being blocked. Please ensure port 389 is accessible between servers.

Once you have the #.new.ldap.com server setup and the required password information, it’s time to enable MMR on your current production system which requires a restart of the LDAP service.

To remove existing LDAP servers from the pool of MMR replica update ldap_master_url and ldap_url on all of your systems including the #.new.ldap.com and excluding #.old.ldap.com.

Note: At this time it is important to take advantage of configuring the cluster for failover. What this means is if the first master or replica listed in the server configuration goes down, then the second master or replica will be used. This will support an upgrade of your LDAP with no downtime. For larger clusters, it is recommended to stagger servers to different masters or replicas to distribute traffic and load. It is also recommended to point read services like Proxy & MTA to replicas and write services like Mailbox to master to improve LDAP performance. The following is an example:

Once the all of the original LDAP servers have been removed from production, you can uninstall LDAP from these systems by running:

By uninstalling each LDAP instance will remove the instance from the cluster.

If the following error occurs when starting the servers on either system, it means there is an issue within the DNS, hostname or zmhostname does not match or a password within zmlocalconfig is not set correctly. Also, if the #.old.ldap.com LDAP is not it will cause this error.

It is time to upgrade all of your #.new.ldap.com servers to Daffodil (v10). All #.new.ldap.com servers need to be upgraded within same maintenance window but one at a time. This will allow you to upgrade the LDAP’s without effecting current production, a partial upgrade will lead to schema mismatch and we recommend scheduling a Maintenance window for 15 minutes for each LDAP server as a pre-cautionary measure. The following steps will need to be performed on each ldap server.

Status: All LDAP servers MMR and replica’s are running Daffodil and all other systems are running the originals version {8815 or 900}

At this time all proxy traffic is passing through the #.old.proxy.com servers and #.new.proxy.com servers are configured but are idle. After completing your testing, Add the #.new.proxy.com and remove #.old.proxy.com from the production traffic.

Once all traffic is successfully passing through your #.new.proxy.com servers, uninstall #.old.proxy.com servers from the cluster by running the following on each of the #.old.proxy.com servers.

/opt/zimbra/libexec/installer/install.sh -u

Note: this command will remove all services on the server that the command is ran on, please verify there are no other Zimbra services on the system before performing the un-installation.

After performing testing, it is time redirect the MTA traffic from the #.old.mta.com’s to the #.new.mta.com’s.

Once all traffic is successfully passing through your #.new.mta.com servers, uninstall #.old.mta.com servers from the cluster by running the following on each #.old.mta.com server.

Copy the files within /opt/zimbra/ssl/zimbra/commercial/ from your #.old.mailbox.com to the #.new.mailbox.com.

Ensure the permissions are set to zimbra on all files.

As root user, execute this command on #.new.mailbox.com:

Repeat these steps until all new mailbox servers have been added.

To configure ssh access to data between all the servers, an ssh key will need to be generated on each server, then each server needs to sync the other servers ssh key. To create ssh keys run zmsshkeygen as a zimbra user on all the servers:

Once the keys are generated, sync the keys by running zmupdateauthkeys on both servers:

Key updates take effect immediately and do not necessitate a server restart. Please refer to guide for more information.

In order for the server statistics to display on the administration console, the syslog configuration files must be modified.

For syslog on Debian or Ubuntu:

For rsyslog:

For rsyslog on RHEL or CentOS:

New installs of {product-abbrev} limit spam/ham training to the first MTA installed. If you uninstall or move this MTA, you will need to enable spam/ham training on another MTA, as one host should have this enabled to run zmtrainsa --cleanup. To do this, set zmlocalconfig -e zmtrainsa_cleanup_host=TRUE.

The migration utility package is available on Zimbra’s public repository.

Following are the steps to download and install the migration utility on Daffodil (v10) mailbox server.

After successful installation, a command zmmodulesport will be available on the server.

Setting up volumes is critical and will be different for each customer, but this is also a time when you can reconsider your volume configuration.

From Daffodil 10.0.5 Patch onwards, support for Unified Storage has been added in the Storage Management module. Unified Storage allows better data management by storing the data from the multiple mailbox servers under the same directory structure in a single S3 bucket.

When using Unified Storage, you can also do a blobless migration if you are using NG HSM’s Centralized Storage feature. In this case, the mail data is not required to be migrated from the S3 bucket and continue to be present at the same location.

Please refer to admin-guide for more details on the feature.

zmmodulesport tool provides the ability to copy your HSM configuration within #.old.mailbox.com into the #.new.mailbox.com and based on storage type, each migration will be different.

Following are the details of the attributes we need to use when migrating the data and the behavior when they are set to TRUE:

Depending upon your Primary and Secondary storage setup, the steps to migrate the HSM data and accounts will vary. Please refer to the following guidelines depending upon your storage setup:

The migration utility only provides the information if the Backup & Restore feature is enabled/disabled for the COS.

Customers who are using NG Delegated Admin can migrate or manually configure admins within Daffodil Admin UI. Since migrating delegated admin configuration is an LDAP attribute, migration only needs to be done once for the full cluster.

If ABQ rules and share folder support for mobile are not being used, you can skip the migration section but please review the device migration section to ensure you are aware of behavior changes after the migration.

Mobile data consists of ABQ rules and share folder configuration for mobile devices and are LDAP attributes that can be migrated before the accounts.

If Zimbra Drive is being used, the data can be exported for all or selected accounts based on the migration schedule. We recommend exporting data prior to moving an account to ensure all the latest files are migrated.

Zimbra Drive for 8.8.15 and 9 stores its data outside of the account. The drive migration tool will migrate Drive Data into the Briefcase of the account for better management and data ownership.

Following are the recommendations when migrating Zimbra Drive data:

A file drive-data.tgz will be created within /opt/zimbra/tmp/modules/drive/.

A file drive-data-batch1.tgz will be created within /opt/zimbra/tmp/modules/drive/.

It’s time to move accounts from #.old.mailbox.com to #.new.mailbox.com using the zmmboxmove command. This command will be used for all accounts (users and systems).

For systems that are using S3 storage, you will need to use --ngMigration flag to update the blob locater once the account has been moved.

After a successful move, the migration script will update the user LDAP to point to #.new.mailbox.com. When the user logs in they will access the data on the new system and if the user is currently logged in, they will receive a message requesting them to reload the browser. For devices using active-sync, when sync is enabled, the clients will receive a re-sync request and will re-sync the account. For all other clients, they should not experience any sync issues.

To obtain a list of all users on a server:

To obtain a list of all calendar accounts on a server:

The zmmboxmove command moves a single mailbox at a time but can be scripted to move

Accounts that exist within the LDAP but haven’t been created will receive the following error when moving:

Update the account manually by updating the zimbraMailHost and zimbraMailTransport attributes by running:

Once all accounts have been migrated, it’s time to update your distribution lists zimbraMailHost with #.new.mailbox.com. Modify and run the following command to update all of your existing DLs

It is recommended to purge user data on the #.old.mailbox.com once all data has been verified and the account is in production on the #.new.mailbox.com. When doing a purge, all data will be deleted on #.old.mailbox.com including drive data. This can be done hours, days, or weeks after the account has been moved based on your business needs, and can be done by running:

If you performed a blobless move, set zimbraMailboxMoveSkipBlobs to FALSE:

For NG Mobile, if you have set the ABQ mode to Strict and if there are any devices updated in the Allowed Device list on NG server, after the mobile data import to the new.server.com server, update the zimbraMobileBlockedDevices to allow syncing of the allowed devices:

Before you start the import, it is recommended to increase the file upload size and auth token lifetime.

The default value of zimbraFileUploadMaxSize is 10MB (10485760) and zimbraAdminAuthTokenLifetime is 12 hour (12h).

If you want to increase the zimbraFileUploadMaxSize value to 100MB, please specify the value in bytes and update the zimbraFileUploadMaxSize attribute on mailbox server.

On COS level, if you want to increase zimbraAdminAuthTokenLifetime value to 24 hours, please specify the value as 24h and update the zimbraAdminAuthTokenLifetime attribute on mailbox server.

Restart mailbox service to make the changes effective:

Once the accounts are moved to #.new.mailbox.com, import Drive data by executing the following on #.old.mailbox.com:

Repeat this on all servers until all accounts have been migrated.