Simple Mail Transfer Protocol (SMTP)
The Simple Mail Transfer Protocol (SMTP) is the de facto standard for transporting mail around the Internet. Anyone who wants to have a mail server capable of sending and receiving mail must be able to support SMTP. As a standards-based protocol (see RFC 5321), SMTP is well understood, platform independent, and well supported across a variety of operating systems and devices.
In this chapter, we’ll discuss the mechanics of SMTP as a protocol and its relationship to other mail-related protocols, such as Post Office Protocol (POP) and Internet Message Access Protocol (IMAP). Then we will go over the Postfix SMTP server, one of the easier and more secure SMTP servers out there.
SMTP defines the method by which mail is sent from one host to another. That’s it. It does not define how the mail should be stored. It does not define how the mail should be displayed to the recipient.
SMTP’s strength is its simplicity, and that is due, in part, to the dynamic nature of networks during the early 1980s (circa when the protocol was invented). Back in those days, people were linking networks together with everything short of bubble gum and glue. SMTP was the first mail standard that was independent of the transport mechanism. This meant people using TCP/IP networks could use the same format to send a message as someone using two cans and a string—at least theoretically.
SMTP is also independent of operating systems, which means each system can use its own style of storing mail without worrying about how the sender of a message stores her mail. You can draw parallels to how the phone system works: Each phone service provider has its own independent accounting system. However, they all have agreed upon a standard way to link their networks together so that calls can go from one network to another transparently.
In the Free Open Source Software (FOSS) world, several software packages (such as Exim, Postfix, Sendmail, and opensmtpd) provide their own implementation of SMTP.
Rudimentary SMTP Details
Ever had a “friend” who sent you an e-mail on behalf of some government agency informing you that you owe taxes from the previous year, plus additional penalties? We’re going to show you how they did it and, what’s even more fun, how you can do it yourself. (Not that we would advocate such behavior, of course.)
The purpose of this example is to show how SMTP sends a message from one host to another. After all, more important than learning how to forge an e-mail is learning how to troubleshoot mail-related problems. So in this example you are acting as the sending host, and whichever machine you connect to is the receiving host.
SMTP requires only that a host be able to send straight ASCII text to another host. Typically, this is done by contacting the SMTP port (port 25) on a mail server. You can do this using the Telnet program. Here’s an example:
Here, the host
mailserver is the recipient’s fictitious mail server. The
25 that follows
mailserver tells Telnet that you want to communicate with the server’s port 25 (standard SMTP port) rather than the normal standard Telnet port 23.
The mail server will respond with a greeting message such as this:
220 mail ESMTP Postfix
You are now communicating directly with the SMTP server.
Although there are many SMTP commands, four are worth noting:
HELO command is used when a client introduces itself to the server. The parameter to
HELO is the hostname that is originating the connection. Of course, most mail servers take this information with a grain of salt and double-check it themselves. Here’s an example:
If you aren’t coming from the example.org domain, many mail servers will respond by telling you that they know your real IP address, but they may or may not stop the connection from continuing.
MAIL FROM: command requires the sender’s e-mail address as its argument. This tells the mail server the e-mail’s origin. Here’s an example:
This means the message is from firstname.lastname@example.org.
RCPT TO: command requires the receiver’s e-mail address as an argument. Here’s an example:
This means the message is destined to email@example.com.
Now that the server knows who the sender and recipient are, it needs to know what message to send. This is done by using the
DATA command. Once it’s issued, the server will expect the entire message, with relevant header information, followed by one empty line, a period, and then another empty line. Continuing the example, firstname.lastname@example.org might want to send the following message to email@example.com:
And that’s all there is to it. To close the connection, enter the
This is the basic technique used by applications that send mail—except, of course, that all the gory details are masked behind a nice GUI application. The underlying transaction between the client and the server remains mostly the same.
Developers of the Postfix mail server implementation wrote the server software from scratch with security in mind. Basically, the package ships in a tight security mode, and it’s up to the individual user to loosen it up as much as is needed for a specific environment. This means the responsibility falls to us (as sysadmins) for making sure we keep the software properly configured (and thus not vulnerable to attacks).
When deploying any mail server, keep the following issues and questions in mind:
• When an e-mail is sent to the server, what programs will it trigger? And under what permissions do those programs run?
• Are those programs securely designed? And can you protect/secure the communication channels between the programs and the end clients via secure protocols?
• If the communication channels cannot be made secure, how can you limit the damage in case of an attack?
Mail service has three distinct components:
• Mail user agent (MUA) The component of the e-mail system that the user sees and interacts with, such as the Thunderbird, Outlook, Evolution, or Mutt program. An MUA is responsible only for reading mail and allowing users to compose mail.
• Mail transport agent (MTA) Handles the process of getting the mail from one site to another. Postfix, Exim, and Sendmail are popular examples of an MTA.
• Mail delivery agent (MDA) Responsible for distributing and sorting any received messages on the local machine to the appropriate user mailbox. The Procmail program is a popular solution for handling the actual mail delivery (MDA) component of e-mail. This is because of its advanced filtering mechanism, as well as its secure design from the ground up.
NOTE Some mail systems integrate all three components. For example, Microsoft Exchange Server integrates the MTA and MDA functionalities into a single system. Postfix, on the other hand, works as an MTA only, passing the task of performing local mail delivery to another external program. This delineation of tasks allows the use of other tools or solutions for tasks such as determining mailbox storage mechanisms.
Installing the Postfix Server
We chose the Postfix mail server in this discussion for its ease of use, simple design, and secure track record. (The author of Postfix also argues that the simplicity has led to improved security.) Postfix provides most of the functionalities that Sendmail program does—in fact, the typical installation procedure for Postfix is to work as a drop-in replacement for Sendmail binaries completely.
Postfix is the default mail server program on most modern Linux distros. In the following sections, we show you how to install Postfix using the built-in package management (Red Hat’s RPM or Debian’s dpkg) mechanism of the distribution. This is the recommended method. We also show how to build and install the software from its source code.
Installing Postfix via DNF in Fedora, CentOS, or RHEL
To install Postfix via DNF on Fedora, CentOS, or RHEL distros, simply use the
dnf package manager as follows:
Once the command runs to completion, you should have Postfix installed.
On older legacy Linux distros, you can use the
chkconfig utility to make sure that your Postfix mail service starts automatically during system boot.
On modern systemd-enabled distros, use the
systemctl command like so:
Finally, you can flip the switch and actually start the Postfix process. With a default configuration, it won’t do much, but it will confirm whether the installation worked as expected.
On systemd-enabled distros, start the postfix service unit via the following:
TIP You may find yourself inheriting or managing an existing Red Hat–based distro that has both of the popular MTAs (Sendmail and Postfix) installed, or you may decide to install a new MTA. If you want to switch the mail subsystem that is running on such a system—from, say, Sendmail to Postfix—you can use the
alternatives facility to switch the default MTA provider. Run the command like so and follow the prompts:
Installing Postfix via APT in Ubuntu
Postfix can be installed in Ubuntu by using Advanced Packaging Tool (APT). Ubuntu does not ship with any MTA software preconfigured and running. You explicitly need to install and set one up. To install the Postfix MTA in Ubuntu, run this command:
You will be prompted to select your Postfix mail server configuration type during the installation process. Here are the available types:
• No configuration This option will leave the current configuration unchanged.
• Internet site Mail is sent and received directly using SMTP.
• Internet with smarthost Mail is received directly using SMTP or by running a utility such as fetchmail. Outgoing mail is sent using a smarthost.
• Satellite system All mail is sent to another machine, called a smarthost, for delivery.
• Local only The only delivered mail is the mail for local users. The system does not need any sort of network connectivity for this option.
We will use the first option, No configuration, on our sample Ubuntu server. The install process will also create the necessary user and group accounts that Postfix needs.
Configuring the Postfix Server
By following the preceding tracks, you have installed the Postfix mail system using your distro’s package manager. After installing the Postfix software, you will need to configure it. Most of its configuration files can be found under the /etc/postfix/ directory.
You configure the server through the /etc/postfix/main.cf configuration file. It’s obvious from its name that this configuration file is the main configuration file for Postfix! The other configuration file of note is the master.cf file. This is the process configuration file for Postfix, which allows you to change how Postfix processes are run. This can be useful, for example, for setting up Postfix on clients so that it doesn’t accept e-mail and forwards to a central mail hub. (For more information on doing this, see the documentation at www.postfix.org.)
Now let’s move on to the main.cf configuration file.
The main.cf File
The main.cf file is too large to list all of its options in this chapter, but we will cover the most important options that will get your mail server up and running. Thankfully, the configuration file is well documented and clearly explains each option and its function.
The sample options discussed next are enough to help you get a basic Postfix mail server up and running at a minimum.
This parameter is used for specifying the hostname of the mail system. It sets the Internet hostname for which Postfix will be receiving e-mail. The default format for the hostname is to use the fully qualified domain name (FQDN) of the host. Typical examples of mail server hostnames are mail.example.com, smtp.example.org, mx1.example.org, and so on. Here’s the syntax:
All e-mail sent from this e-mail server will look as though it came from this parameter. You can set this to either
$mydomain, like so:
Notice that you can use the value of other parameters in the configuration file by placing a
$ sign in front of the variable name.
This parameter lists the domains that the Postfix server will take as its final destination for incoming e-mail. Typically, this value is set to the hostname of the server and the domain name, but it can contain other names, as shown here:
You can run the Postfix server in two modes of delivery: directly to a user’s mailbox or to a central spool directory. The typical way is to store the mail in /var/spool/mail. The variable will look like this in the configuration file:
The result is that mail will be stored for each user under the /var/spool/mail directory, with each user’s mailbox represented as a file. For example, e-mail sent to firstname.lastname@example.org will be stored in /var/spool/mail/yyang.
mynetworks variable is an important configuration option. This lets you configure what servers can relay through your Postfix server. You will usually want to allow relaying from local client machines and nothing else. Otherwise, spammers can use your mail server to relay messages. Here’s an example value of this variable:
If you define this parameter, it will override the
mynetworks_style parameter. The
mynetworks_style parameter allows you to specify any of the keywords
host. These settings tell the server to trust these networks to which the server belongs.
CAUTION If you do not set the
$mynetworks variable correctly and spammers begin using your mail server as a relay, you might quickly find a surge of angry online mail administrators e-mailing you about it. Furthermore, it is a fast way to get your mail server blacklisted by one of the spam control techniques, such as a DNS Blacklist (DNSBL) or Realtime Blackhole List (RBL). Once your server is blacklisted, very few people will be able to receive mail from you, and you will need to jump through a lot of hoops to get unlisted. Even worse, no one will tell you that you have been blacklisted.
This variable allows you to return a custom response when a client connects to your mail server. It is a good idea to change the banner to something that doesn’t give away what server you are using. This just adds one more slight hurdle for hackers trying to find faults in your specific software version.
This parameter specifies the network interface addresses that the mail system receives mail on. The default behavior is for the Postfix server software to make use of all active interfaces on the machine when accepting connections. Its default value is
all. Setting this value to
ipv6 will make Postfix support IPv6. Here are some example values that this parameter accepts:
Tons of other parameters in the Postfix configuration file are not discussed here. You might see them commented out in the configuration file when you set the preceding options. These other options will allow you to set security levels and debugging levels, among other things, as required.
Now let’s move on to running the Postfix mail system and maintaining your mail server.
Checking Your Configuration
Postfix includes a nice tool for checking a current configuration and helping you troubleshoot it. Simply run the following:
This will list any errors that the Postfix system finds in the configuration files or with permissions of any directories that it needs. A quick run on our sample system shows this:
Looks like we made a typo in the configuration file!
When going back to fix any errors in the configuration file, you should be sure to read the error message carefully and use the line number as guidance, not as absolute. This is because a typo in the file could mean that Postfix detected the error well after the actual error took place.
In this example, an error of omission (forgetting the = symbol) that we made on line 83 in the configuration file was shown as occurring in lines 83 through lines 115 due to how the parsing engine works. However, by carefully reading the error message, we knew the problem was with the “mydomain” parameter, and so it took only a quick search before we found the real line culprit.
Let’s run the check again:
Groovy! No errors this time. We’re ready to start using Postfix.
TIP You can use the nifty little postconf utility to quickly query or display the value of parameters in your Postfix configuration files. For example, to view the current value of the
mydomain parameter, you can run the following:
$ postconf mydomain
mydomain = localdomain
Running the Server
Controlling the Postfix mail server is easy and straightforward. On systemd-enabled distros, just pass the correct
start/stop/reload option to the
systemctl utility and specify the postfix service unit. To start Postfix, type the following:
When you make any changes to the configuration files, you need to tell Postfix to reload itself to make the changes take effect. Do this by using the
Make sure that Postfix is configured to automatically start up between reboots by typing the following:
Checking the Mail Queue
Occasionally, the mail queues on your system will fill up. This can be caused by network failures or other various failures, such as other external mail servers. To check the mail queue on your mail server, simply type the following command:
This command will display all of the messages that are in the Postfix mail queue. This is the first step in testing and verifying that the mail server is working correctly.
Flushing the Mail Queue
Sometimes after an outage, mail will be queued up, and it can take several hours for the messages to be sent. Use the
postfix flush command to flush out any messages that are shown in the queue by the
The newaliases Command
The /etc/aliases file contains a list of e-mail aliases. This is used to create site-wide e-mail lists and aliases for users. Whenever you make changes to the /etc/aliases file, you need to tell Postfix about it by running the
newaliases command. This command will rebuild the Postfix databases and inform you of how many names have been added.
Making Sure Everything Works
Once the Postfix mail server is installed and configured, you should test and test again to make sure that everything is working correctly. The first step in doing this is to use a local mail user agent, such as pine (text-based, freeware), mutt (text-based, GNU GPL), or mailx (simple command-line MUA for UNIX systems), to send e-mail to yourself. If this works, great; you can move on to sending e-mail to a remote site, while keeping an eye on the output of the
mailq command to see when the message gets sent. The final step is to make sure you can send e-mail to the server from outside networks (that is, from the Internet). If you can receive e-mail from the outside world, your work is done.
On Fedora, RHEL, and CentOS systems, by default, mail logs go to /var/log/maillog, as defined by the rsyslogd configuration file. If you need to change this, you can modify the rsyslogd configuration file, /etc/rsyslog.conf, by editing the following line:
Most sites run their mail logs this way, so if you are having problems, you can search through the /var/log/maillog file for any relevant messages.
Debian-based systems, such as Ubuntu, store the mail-related logs in the /var/log/mail.log file.
openSUSE and SUSE Linux Enterprise (SLE) store their mail-related logs in the files /var/log/mail, /var/log/mail.err, /var/log/mail.info, and /var/log/mail.warn.
If Mail Still Won’t Work
If mail still won’t work, don’t worry. SMTP isn’t always easy to set up the first time. If you still have problems, walk logically through all of the steps and look for errors. The first step is to look at your log messages, which might show that other mail servers are not responding. If everything seems fine there, check your Domain Name System (DNS) settings. Can the mail server perform name lookups? Can it perform Mail Exchanger (MX) lookups? Can other people perform name lookups for your mail server? It is also possible that e-mails are actually being delivered but are being marked as junk or spam at the recipient end. If possible, ask the receiver to check the junk or spam mail folder at their end.
Proper troubleshooting techniques are indispensable for good system administration. A good resource for troubleshooting is to look at what others have done to fix similar problems. Check the Postfix web site at www.postfix.org, or search online, for the problems or symptoms of what you might be seeing.
In this chapter, you learned the basics of how SMTP works. You also installed Postfix and learned how to configure a basic Postfix mail server. With this information, you have enough knowledge to set up and run a minimal production mail server.
If you’re looking for additional information on Postfix, start with the online documentation at www.postfix.org. The documentation is well written and easy to follow. It offers a wealth of information on how Postfix can be extended to perform a number of additional functions that are outside the scope of this chapter. Another excellent reference on the Postfix system is The Book of Postfix: State-of-the-Art Message Transport, by Ralf Hildebrandt and Patrick Koetter (No Starch Press, 2005). This book covers the Postfix system in excellent detail.
As with any other service, don’t forget to keep up with the latest news on Postfix. Security updates do come out from time to time, and it is important that you update your mail server to reflect these changes.