Carbonio User Mail Replica

Carbonio User Mail Replica#

The Carbonio architecture is mostly based on services that make nodes stateless, redundant, and clustered by design. The only stateful service is the Mailstore, due to the fundamental role it plays in storing metadata, binary blobs, and connection cache.

While this situation could represent a single point of failure, a replication mechanism Carbonio can be added, that drastically increases the availability of the Mailstore service.

How it works#

User Mail Replica is the foundation of the mechanism described above, which is an account-based, real-time replication mechanism that allows Carbonio to keep multiple instances of a mailbox within different Mailstores.

The User Mail Replica part is in charge of encoding and transmitting all the transactions of the account. Once processed by the User Mail Replica, the events are consumed by one agent, or even by multiple agents, in the destination Mailstore.

User Mail Replica Requirements#

There are two requirements to satisfy to be able to install the User Mail Replica.

  1. The Carbonio subscription must include the User Mail Replica module. The User Mail Replica is licensed “for enabled accounts”. The license can be verified with command

    zextras$ carbonio core getLicenseInfo | grep -e ZxHA -e ha_basic -A2

         ZxHA
             quantity                                                    1000
             licensed                                                    true
--
             name                                                        ha_basic
             quantity                                                    1000
             enabled                                                     true

  2. All the primary volumes of the mailbox must be configured as Centralized Storage.

Enabling User Mail Replica#

To enable User Mail Replica you need to configure the endpoints of all the streamer nodes, using either their IPs or FQDNs, which are supposed to expose port 9092 reachable from each of the other Mailstores.

Example

Suppose the infrastructure uses the following nodes

  • event1.example.com with IP 10.0.10.11

  • event2.example.com with IP 10.0.10.13

  • event3.example.com with IP 10.0.10.14

Either of the following commands must be executed

zextras$ carbonio config set global brokers "event1.example.com:9092,event2.example.com:9092,event3.example.com:9092"

Or

zextras$ carbonio config set global brokers "10.0.10.11:9092,10.0.10.12:9092,10.0.10.13:9092"

To verify that the settings have been applied and the service operates correctly, you can use the commands presented in section User Mail Replica below.

User Mail Replica Usage#

A number of CLI commands can be used to carry out routine operations with the User Mail Replica: initialise, monitor, promote, and delete a User Mail Replica.

Limitations of the Commands#

The command presented in this section do not support:

  • regular expressions in the account name: john.doe@example.com is supported, while john*@example.com or ?ohn@example.com are not

  • distribution lists

User Mail Replica Initialisation#

To replicate a mailbox to another Mailstore you can use the setAccountDestination command, which needs as parameters

  • the destination Mailstore’s FDQN (e.g., mailstore1.example.com)

  • the priority of the nodes. This information can be used in case the same account has been replicated more than once, to identify the first to be used. A lower value means a higher priority (e.g., a User Mail Replica with value 10 has a higher priority than User Mail Replicas with values 11, 20, or 100)

  • the account to replicate. Multiple accounts are also available, either comma separated on the command line or from an input file, with one account per line. In the remainder, we call this file /tmp/accounts, which consists of two lines:

Example of valid commands are:

zextras$ carbonio MailReplica setAccountDestination mailstore1.example.com 10 accounts user1@customer.tld,user2@customer.tld

zextras$ carbonio MailReplica setAccountDestination mailstore1.example.com 10 input_file /tmp/accounts

The Global Administrator will receive a notification as soon as the User Mail Replica initialisation is completed.

User Mail Replica Monitoring#

To monitor the status of a User Mail Replica, you can use the getAccountStatus command and refine the output by providing either of the following parameters:

  • mailHost, to verify the status of all the replicated accounts active in the source mailstore

  • replicaServer, to verify the status of all the accounts replicated on a specific mailstore

  • accounts, to limit the list to a (comma separated) subset of accounts

  • domains, to limit the list to all the replicated accounts of one ore more (comma separated) domains

  • accountStatus, to list only accounts with active or paused replica on the source Mailstore

  • replicaStatus, to list only accounts with available or unavailable replica on the destination Mailstore

Without any parameter, the command will show the status of all the accounts configured for the User Mail Replica. For each account, the output reports:

accountId            eg. 9e94f5e0-8e0d-4f61-93aa-00747ac3dba6
accountName          eg. user@demo.zextras.io
accountMailHost      eg. mbox1.demo.zextras.io
paused               true|false
mailboxId            mailboxId of the local MariaDB
sequence             value of sequence in the local MariaDB (on the source)
itemId               value of highest itemId in the local MariaDB (on the source)

Then, for each replica:

replicas
accountId         eg. 9e94f5e0-8e0d-4f61-93aa-00747ac3dba6
itemId            value of highest itemId in the local MariaDB (on the replica)
sequence          value of sequence in the local MariaDB (on the replica)
mailboxId         mailboxId of the local MariaDB (on the replica)
destination       address of the current mailstore that host the replica
available         true|false
priority          prority of the replica
canConnectToKafka status of connection to kafka

User Mail Replica Promotion#

The architecture of User Mail Replica allows for a quick promotion of a replica Node at any time. Indeed, since all the metadata are synchronously replicated in the event queue and the blobs are stored in the centralised volume, the Administrator can trigger the promotion even if the source Mailstore is offline (e.g., the Mailstore is in maintenance mode, there is a hardware failure o a network problem, and so on).

To promote an account, Administrators can use the promoteAccounts command and refine the output by providing either of the following parameters:

  • accounts, to promote one or more (comma separated) accounts, using the first User Mail Replica (lowest priority)

  • input_file, to promote accounts for a file (one per line), using the first User Mail Replica (lowest priority)

  • source_mail_host, to promote all the accounts hosted by a specific Mailstore

Example of valid commands are:

  • Promote accounts

    zextras$ carbonio MailReplica promoteAccounts accounts alice.doe@example.com,bob.doe@example.com

  • Promote accounts stored in a file

    zextras$ carbonio MailReplica promoteAccounts input_file /tmp/accounts

  • Promote all accounts on a mailstore

    zextras$ carbonio MailReplica promoteAccounts source_mail_host mbox1.example.com

Global Admin will receive a notification as soon as the User Mail Replica promotion is completed.

User Mail Replica Deletion#

The Administrator can delete the replicated metadata anytime, using the removeAccountDestination command, by providing either of the following parameters:

  • destination, to specify which is the Mailstore from where you want to delete the replica

  • accounts, also multiple (comma separated) accounts or an input file (with multiple accounts, one per line), to specify which account metadata must be deleted

Contents