Mattermost self-hosting tutorial

Intro

In a recent post I've talked about Nextcloud, a Free and Open Source file-hosting solution. Today I'll describe how to self-host Mattermost, an Open Source instant messaging system, similar to commercial services such as Slack and Microsoft Teams.

I'll be hosting Mattermost on a DigitalOcean server, although things shouldn't be much different with other cloud providers or if you want to use a local machine instead.

This might not be relevant for you, but if you are migrating from another Mattermost instance, I'll also explain how to transfer the old data over.

Cloud server provisioning

Let's start by spinning up a cloud server with some pre-installed packages and some basic initial configuration.

DigitalOcean account

You'll need a DigitalOcean account if you want to create an actual Mattermost instance as you follow this guide. Note that you'll be charged for creating DigitalOcean machines (or droplets, in DigitalOcean jargon) and using other of their services. Spinning up a new DigitalOcean machine is very easy, make sure you cancel any unused instances to avoid extra costs.

Most DigitalOcean operations can be launched via a command-line tool called doctl. This guide assumes doctl is installed and that the DIGITALOCEAN_ACCESS_TOKEN environment variable is set to a valid DigitalOcean API token.

export DIGITALOCEAN_ACCESS_TOKEN=`pass show digitalocean.com/token`

In order to provision a new machine, first define the SSH key that you intend to use to connect to it. The list of the SSH keys available for a DigitalOcean account can be retrieved with the following command.

doctl compute ssh-key list

You also need to specify a system image. The list of available images can be retrieved with doctl compute image list or found at this page. Use a recent Ubuntu, e.g. ubuntu-24-04-x64.

Finally, you'll need a domain name to assign to Mattermost once installed. Choose a domain and make sure you have access to its DNS settings.

Machine pre-configuration with cloud-init

We'll be using cloud-init to apply some initial configuration to the newly created machine, like installing some packages and setting up a firewall. Prepare a cloud-init configuration file like the one below, you'll need it when provisioning the machine (see the --user-data-file option). Note that the file has to start with #cloud-config to be interpreted properly.

#cloud-config
package_update: true
package_upgrade: true
package_reboot_if_required: true
packages:
  - certbot
  - mattermost
  - nginx
  - postgresql
  - python3-certbot-nginx
apt:
  sources:
    mattermost.list:
      source: "deb https://deb.packages.mattermost.com $RELEASE main"
      keyid: A1B3 1D46 F0F3 A10B 02CF  2D44 F8F2 C317 4477 4B28
      keyserver: hkp://keyserver.ubuntu.com:80
write_files:
  - path: /etc/nftables.conf
    content: |
      flush ruleset
      table inet firewall {
          chain inbound {
              type filter hook input priority 0; policy drop;
              iif lo accept
              meta l4proto {icmp, ipv6-icmp} accept
              ct state vmap {
                  established: accept, related: accept, invalid: drop
              }
              ct state new limit rate over 1/second burst 10 packets drop
              tcp dport 80 accept
              tcp dport 443 accept
              tcp dport ssh accept
          }
          chain forward {
              type filter hook forward priority 0; policy drop;
          }
      }
runcmd:
  - [ systemctl, enable, nftables.service ]
  - [ systemctl, restart, nftables.service ]

Provisioning

A new DigitalOcean machine can now be provisioned with this command.

export region=lon1
export size=s-1vcpu-2gb
export image_id=ubuntu-24-04-x64
export ssh_key_id=00000000
doctl compute droplet create \
    --enable-ipv6 \
    --image "${image_id}" \
    --region "${region}" \
    --size "${size}" \
    --ssh-keys "${ssh_key_id}" \
    --user-data-file build/cloud-config \
    --wait \
    mattermost

After a short while, the command should return a success message that includes some machine details, such as its IPv4 and IPv6 addresses.

Connect to the machine from the DigitalOcean web console and retrieve its SSH public key fingerprint with ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.

From a terminal, SSH into the machine as root, check that the SSH fingerprint corresponds to what obtained from the web console.

Once logged in, verify that the firewall has been activated correctly with nft list ruleset. The output should be similar to what was defined in our cloud-config file.

Initial setup

If you have used the cloud-init configuration as suggested at the beginning of this guide, Mattermost, PostgreSQL, and Nginx should be already installed on your server. We will now need to configure each of these services, as well as create a PostgreSQL database and request a TLS certificate from Let's Encrypt.

PostgreSQL

SSH into the server as root. Connect to PostgreSQL with sudo --user=postgres psql, then run these commands:

CREATE DATABASE mattermost;
\connect mattermost;
CREATE USER mmuser WITH PASSWORD 'PASSWORD';
GRANT ALL PRIVILEGES ON DATABASE mattermost to mmuser;
ALTER DATABASE mattermost OWNER TO mmuser;
GRANT USAGE, CREATE ON SCHEMA PUBLIC TO mmuser;

Do not forget to replace PASSWORD with a strong password. Edit your Postgres' pg_hba.conf to allow local connections. In practice, this means replacing:

local   all             all                        peer
host    all             all         127.0.0.1/32   scram-sha-256
host    all             all         ::1/128        scram-sha-256

With:

local   all             all                        trust
host    all             all         127.0.0.1/32   trust
host    all             all         ::1/128        trust

Reload Postgres with systemctl reload postgresql.

It should be now possible to connect to the database with the credentials used above:

psql --dbname=mattermost --username=mmuser --password

For further details, look at the Mattermost documentation.

Mattermost

If started automatically already, stop the Mattermost service with systemctl stop mattermost. Save the following configuration in /opt/mattermost/config/config.json, then restart the service.

{
    "ServiceSettings": {
        "SiteURL": "https://mattermost.example",
        "EnableMultifactorAuthentication": true,
        "EnforceMultifactorAuthentication": true,
        "EnableEmailInvitations": true,
        "EnableCustomBrand": true,
        "CustomBrandText": "Acme Corp"
    },
    "SqlSettings": {
        "DriverName": "postgres",
        "DataSource": "postgres://mmuser:PASSWORD@localhost:5432/mattermost"
    },
    "EmailSettings": {
        "SendEmailNotifications": true,
        "RequireEmailVerification": true,
        "FeedbackName": "No-Reply",
        "FeedbackEmail": "mattermost@example.com",
        "ReplyToAddress": "info@example.com",
        "EnableSMTPAuth": true,
        "SMTPUsername": "mattermost@example.com",
        "SMTPPassword": "PASSWORD",
        "SMTPServer": "mail.example.com",
        "SMTPPort": "465",
        "ConnectionSecurity": "TLS",
        "SendPushNotifications": true,
        "PushNotificationServer": "https://push-test.mattermost.com/"
    }
}

It might be necessary to adjust the file's permissions and ownership:

chmod 600 config.json
chown mattermost:mattermost config.json

Nginx

First, choose a domain name, e.g. mattermost.example, and configure its DNS to point to the server.

Overwrite the Nginx default configuration /etc/nginx/sites-enabled/default with:

server {
    listen 80 default_server;
    server_name mattermost.example;
}

Use certbot to get a TLS certificate from Let's Encrypt:

certbot \
    --agree-tos \
    --domain mattermost.example \
    --email info@example.com \
    --non-interactive \
    --nginx

Now overwrite the Nginx configuration again with:

upstream backend {
    server localhost:8065;
    keepalive 32;
}

server {
    listen 80 default_server;
    listen [::]:80 default_server;
    server_name mattermost.example;
    return 301 https://$server_name$request_uri;
}

server {
    listen 443 ssl;
    listen [::]:443 ssl;
    server_name mattermost.example;

    ssl_certificate /etc/letsencrypt/live/mattermost.example/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/mattermost.example/privkey.pem;
    include /etc/letsencrypt/options-ssl-nginx.conf;
    ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

    add_header Strict-Transport-Security max-age=15768000;
    ssl_stapling on;
    ssl_stapling_verify on;

    location ~ /api/v[0-9]+/(users/)?websocket$ {
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        client_max_body_size 50M;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Frame-Options SAMEORIGIN;
        proxy_buffers 256 16k;
        proxy_buffer_size 16k;
        client_body_timeout 60s;
        send_timeout 300s;
        lingering_timeout 5s;
        proxy_connect_timeout 90s;
        proxy_send_timeout 300s;
        proxy_read_timeout 90s;
        proxy_http_version 1.1;
        proxy_pass http://backend;
    }

    location / {
        client_max_body_size 100M;
        proxy_set_header Connection "";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
        proxy_set_header X-Frame-Options SAMEORIGIN;
        proxy_buffers 256 16k;
        proxy_buffer_size 16k;
        proxy_read_timeout 600s;
        proxy_http_version 1.1;
        proxy_pass http://backend;
    }
}

Restart Nginx with systemctl restart nginx. If this doesn't work, you might need to kill Nginx with pkill -f nginx & wait $!, then start it with systemctl start nginx. You should now be able to connect to https://mattermost.example.

Certbot should have created a systemd timer that takes care of automatic certificate renewals. Run systemctl list-timers to confirm. The renewal process can be tested with certbot renew --dry-run.

Data migration

If you're migrating from another Mattermost instance, this is the time to transfer all your data over, i.e. users and chat history. We assume that the old instance was based on the same Mattermost version and on the same database, PostgreSQL.

You will need:

a dump of the old PostgreSQL database,
a copy of the old Mattermost data folder,
a copy of the old Mattermost configuration file config.json.

Stop Mattermost with systemctl stop mattermost. Load the old database with:

psql \
    --dbname=mattermost \
    --username=mmuser \
    --password < dump.sql

Depending on your circumstances, you might want to consider the following extra options: --no-psqlrc, --set ON_ERROR_STOP=on, and --single-transaction.

If you're migrating from an older PostgreSQL version, connect to the database with sudo --user=postgres psql and run this command:

\connect mattermost;
ANALYZE VERBOSE;

This is important for performance reasons. See the Mattermost docs for details.

Copy the old data folder over to /opt/mattermost/data/. Make sure the files belong to user mattermost and group mattermost.

Compare the old config.json with the new one and see if there's anything that needs to be adjusted. Finally, restart the service with systemctl start mattermost.

Backups

You most certainly want to have a backup mechanism in place for disaster recovery. You will need to backup:

the PostgreSQL database,
the Mattermost data folder,
the Mattermost configuration (config.json),
the Nginx configuration and TLS certificates.

You might also want to keep note of the exact Mattermost, Nginx, and PostgreSQL versions used.

Most of the above can be taken care of by simply enabling DigitalOcean's instance backup mechanism. This will increase your bill, but it's probably a good investment.

For the database backup, however, you'll need to use something specific like pg_dump. Without pg_dump (or a similar tool) there's a risk to end up with inconsistent snapshots, e.g. if they get taken while a database transaction is in progress.

Put this, or a variation thereof, in a cron script or in a systemd timer. The dumped file will be then backed up as part of the DigitalOcean instance backup mechanism. Optionally, you can add a call to a health check service so as to get a notification if the backup script stops working.

#!/bin/bash

# A dummy script to take a backup copy of Mattermost's database.

set -euo pipefail
sudo --user=postgres pg_dump --dbname=mattermost > /root/pg_dump.sql
curl --max-time 10 --retry 5 https://healthcheck.example.com/healthcheck-id

Todo

At this point it'd be good to set up a fail2ban service to detect and block malicious login attempts. This is left as an exercise for the reader.

Depending on your context, in addition or as an alternative to fail2ban, you should consider putting the whole Mattermost service behind a virtual private network. If you're so inclined, WireGuard is an excellent tool.

Outro

As always, this post is mostly a note-to-self - although I do hope someone else might find it useful. Should you find anything wrong or incomplete and you want to let me know, that'd be very appreciated. Until next!