Recently, Google started to send notices saying that I am approaching my storage limit (15 GB). Apparently the bulk of my usage is e-mails.

To address this I have set up IMAP server as an e-mail archive target, copied all of my email to said archive target, and will eventually delete old emails, freeing up space.

This will be done with Dovecot as the IMAP server and imapsync will be the utility to copy all of the e-mail from Gmail into our archive target.

Installing and Configuring Dovecot

To install and configure Dovecot, I created an Ansible role: ansible-role-dovecot-archive. This role installs and configures Dovecot with virtual users and some mailbox adjustments to better support Gmail sources.

To add virtual users, generate a password with doveadm pw and add it to /etc/dovecot/users. My usernames are the email address I am looking to archive.

source@gmail.com:{CRYPT}$2y$05$b/E5k9b98ZPI5FqVgyfkju9j6uv6EEqTBtfDSeiCboHgVZHCIYjFW:5000:5000::

Copying E-mails with imapsync

To copy emails from Gmail to our Dovecot server, I used imapsync. The developers have a handy Gmail-specific FAQ: https://imapsync.lamiral.info/FAQ.d/FAQ.Gmail.txt

If the source email account is a gmail.com Gmail account, we need would need to obtain an OAuth2 access token.

Otherwise, we would need to generate an app password.

gmail.com accounts

For personal/gmail.com Gmail accounts, we have to obtain an OAuth2 access token to use with imapsync. The imapsync team provides a oauth2_imap utility to facilitate obtaining an access token.

See https://imapsync.lamiral.info/oauth2/oauth2_imap/README_oauth2.txt for more information

Google Workspace Gmail

If your Gmail account is a Google Workspace Gmail account, the process is a little simpler. We generate an app password for imapsync to authenticate to Gmail.

Copy all the emails

Now that we have the pieces in place, we first test things out with the --dry and --justfolders options.
The following imapsync invocation assumes that we have an app password that is passed in via the IMAPSYNC_PASSWORD1 and IMAPSYNC_PASSWORD2 environment variables.
If we are working with an OAuth2 token, we specify --oauthaccesstoken1 with the path to the txt file from oauth2_imap.

$ imapsync --user1 source@gmail.com \
  --dry --justfolders \
  --host2 dovecot-archive.localdomain --user2 source@gmail.com \
  --gmail1 --noskipcrossduplicates \
  --exclude 'Important|Spam|Trash' \
  --addheader \
  --delete2

You may find that some labels may not appear in the output. If we want those labels, we need to go to Settings (Gear icon) > Labels. Check the boxes next to Show in IMAP and rerun the above until things appear right.

One gotcha I ran into was that some labels had periods in them (e.g. example.com), so they were replaced with underscores in Dovecot (i.e. example_com). A mailing list discussion mentions a few options such as the fs layout or the Listescape plugin. I have decided to leave things as-is for now.

The above command, is the result of a few iterations and troubleshooting (because of the missing labels). I have opted to exclude Important, Spam, and Trash as I do not want those mailboxes in my archive. Some of them you could exclude in Gmail by unchecking the Show in IMAP like we did for the “missing” labels.

After some iterations, we remove the --dry and --justfolders and copy the e-mails (for real). For my main e-mail account, I started this in the late afternoon and it finished sometime overnight.

Validation

To validate that my e-mails have been copied over, I connected to Dovecot using Thunderbird and browsed around. One quick verification check were e-mails that had image attachments. So far, things look good.

When adding the account to Thunderbird, we fill out the form and click Configure manually. There are DNS records that can be added to facilitate autodiscovery, but is an exercise for another day.

Clicking Configure manually will show the Manual configuration section where we will enter the Hostname for both the Incoming Server and the Outgoing Server. Since we are not sending e-mail, Outgoing Server does not matter too much but we will make it the same for simplicity.

At this point, we click Advanced config to finish configuration.

In the Server Settings section of the new account, under Security Settings, we want to Fetch Certificate and then Add Exception such that Thunderbird trusts the self-signed certificate from Dovecot.

Next Steps

More Testing

This is still early days. Since writing this, I have identified a few gaps that required some adjustments such as personal Gmail vs Google Workspace Gmail and emails that appear to be missing. More testing and more documenting.

Debian 13 Support

The Ansible role works on Debian 12 Bookworm (and older). This is because with Debian 13 Trixie, Dovecot was upgraded to v2.4 bringing major configuration changes that has to be looked at first.

Automation

Automation is another thing to consider. Since I am copying emails from live accounts and deleting emails, we do not necessarily want all deletes to propagate to our archive. This means that the automation is not as straightforward as we like for it to be. One idea is to run imapsync twice:

• the first run would exclude the All Mail folder and delete emails that do not exist in Gmail
• the second run would only synchronize emails in the All Mail folder, but not delete anything

This should (probably) work, but testing is required.