A full email server how-to
This guide will entail setting up an entire email system on a server with a public internet address that is assumed to be up continuously. If your server is not up continuously or you want failover capability, you can setup postfix satellite servers, which I will cover in the guide Installing a Postfix Satellite Server.
Since the setup of a mail server is a bit complicated and complex to understand, especially for beginners, I will first begin with descriptions of the software and how they link together. I would also like to lay out two separate guides: The first guide will lay out spam configuration settings: Anti-spam control will be integrated with my installation guide, but this separate guide will go into details on what these settings do and how you can customize them to better suit your needs and environment. The second guide will be aimed at dovecot's sieve plugin and how you can use it with IMAP to automatically sort mail on your server. This works in conjunction with anti-spam in the the first guide, so it is a very useful tool.
This guide is not yet completed, as of 12/31/19.
Overview of Dovecot and Postfix
To start off with, Dovecot is a IMAP and POP3 server. In this guide, we will be configuring it as an IMAP server. The reason I prefer an IMAP server over POP3 is because IMAP gives you the ability to access email on multiple devices such as your laptop, your desktop, your phone, your tablet, and see the same email view everywhere. The client synchronizes with the server (and phones can even receive push notifications) so if you receive an email on your phone while on the go, read it, and then move it to another folder, when you get home, your email will be in the same folder you put it on your phone. If you delete the email from the folder on your desktop, and then open that folder on your phone, the email will be gone on your phone as well. That is the magic of IMAP. Think of it like your own GMail server.
Postfix is Mail Transport Agent (MTA). It uses the Simple Mail Transport Protocol (SMTP) to send mail around the internet. When you send mail from your email client, you are connecting to an SMTP server. The SMTP server then connects to other SMTP servers until it connects to the destination SMTP server responsible for the user who owns the email account you are sending mail to. At that point the SMTP server transfers the mail to a spool on the server. When the user who owns the email account reads their email, the email server (such as Dovecot) reads from the spool to display the email to the user. That's the short end of the stick.
Figure 1 is a simplified diagram of postfix and dovecot. In this figure is shown a server running the dovecot and postfix daemons on a public IP on the internet. A user is connected with his/her client of client of choice. The user sends mail via SMTP by connecting to postfix. The user can read and manage mail folders through their mail client via IMAP by connecting to the Dovecot daemon over IMAP.
In Figure 2, I expand on how the configuration is ultimately achieved. Mostly, you will notice the addition of anti-spam components and sieve. When a message is received, the postfix configuration is setup to run the message through a DNS blacklist. This blacklist ensures that the message is received from a valid, trusted domain. If that domain is not trusted by authoritative sources, it is considered spam and postfix will reject the message. You can configure which DNS blacklists you use on your server (and it does make a difference in anti-spam performance). This is the first level of anti-spam control. As well, there are a number of additional controls you can choose to add to your postfix configuration to help stop spam. Those are not shown here. Next, the message is handed off to spamassassin via a TCP socket (spamassassin actually runs as a daemon). Spamassassin scores the message via its algorithm to determine if the message is spam. The configuration of postfix we set tells spamassassin to forward the resulting mail to dovecot. This is necessary because dovecot's support for plugins will not work if we let postfix drop the mail directly into the spooler. Sieve is a dovecot plugin (which we must configure when installing our mail server).
I recently looked up some information on dovecot and spam. Turns out, there is a very good resource on configuring the anti-spam controls within dovecot. Rather than re-posting all of the relevant information, I provide links to the content. These guides are very well written, and while they are aimed at users of Ubuntu 18.04, they work well with Debian (which is essentially Ubuntu without a GUI). Take a look:
Part 4: How to Set up SPF and DKIM with Postfix on Ubuntu Server - This guide shows you how to enable SPF and DKIM for your mail server and domain. This guide, and the resources provided here for DMARC, allows you to communicate to other email servers that you are a legitimate mail server. This prevents email sent from your domain from landing in the spam folder of the recipient (or from not being delivered at all).
7 Effective Tips for Blocking Email Spam with Postfix SMTP Server - This guide is an essential resource for configuring basic anti-spam features of dovecot and postfix. This guide should be taken as a supplement, as there are additional anti-spam controls available which are not discussed here.
Block Email Spam with Postfix and SpamAssassin Content Filter - This guide shows the installation of SpamAssassin. This guide provides a single method of configuring SpamAssassin, however it should be noted that there are a few ways to enable SpamAssassin integration with your mail servers.
About DMARC (from Google)