Silver Bullets

Vulnerability testing SMTP with my postfix-dkim image

Building on the Postfix-dkim container, receiving email as described in the preceding "Receiving email with postfix", I examine the vulnerabilities of this configuration.

I took months getting confident enough with SSH security to trust myself with VPSs. Here, I'll examine the principals of SMTP, vulnerabilities, and tools, online, and off, to assess security.

Whilst the test target here is firmly set as my docker image, the principles and procedures presented enable the security of any mail server to be quickly verified.

Testing vulnerabilities

Spoofing and open relay are the major vulnerabilities to watch out for. Encryption is important, but it directly effects clients more than the integrity of the MTA service.

Testing these may be considered anti-social. Even if you own, or rent, both endpoints, service providers often stipulate that you should not do, what might be construed as, vulnerability exploits on their network. That's why this post is long, I don't want anyone to have to repeat it for lack of detail.

Spoofing using single line sendmail

This must be performed from a separate VPS. I couldn't recreate using the two containers on the same host.

I can set any from address I like when sending to this server:

echo -e "From: serioussam@example.com\nSubject: you see this\nThis was my sendmail one-liner.\n" | sendmail -v mailbox@silverbullets.co.uk

It will be accepted and, superficially, appear to come from whosoever I choose. The first line of the header (viewed using mail) contains a different "From", without trailing colon, "field" followed by the real username@hostname and date.

The passed "From" argument is in Return-Path, visible in the header/full information of larger email providers. Received: may be followed with a "from" or "by"; which aren't amenable to spoofing.

I'm not pasting the received email, it is bloated and distracting.

Spoofing using telnet

This is the entirety of the email I see on my receiving server as a result of the telnet session I will describe next:

From serioussam@example.com  Thu Oct 27 20:20:14 2022
Return-Path: <serioussam@example.com>
X-Original-To: mailbox@silverbullets.co.uk
Delivered-To: mailbox@silverbullets.co.uk
Received: from vps1.silverbullets.co.uk (vps1.silverbullets.co.uk [54.38.15.102])
    by 83c72e74a80a (Postfix) with ESMTP id 0D5DA103EA3
    for <mailbox@silverbullets.co.uk>; Thu, 27 Oct 2022 20:19:45 +0000 (UTC)
Authentication-Results:83c72e74a80a; dkim=permerror (bad message/signature format)
Subject: Re: this is across the internet

spoofing the sender

Authentication-Results show that DKIM failed. That probably is unacceptable to major email providers, I am not about to try and telnet into those to find out!

This is a truncated version of what I sent using telnet:

anotheruser@vps1:~$ telnet silverbullets.co.uk 25
Trying 107.175.0.0...
Connected to silverbullets.co.uk.
Escape character is '^]'.
220 83c72e74a80a ESMTP Postfix (Debian/GNU)
ehlo vps1.silverbullets.co.uk
250-83c72e74a80a
mail from: serioussam@example.com
250 2.1.0 Ok
rcpt to: mailbox@silverbullets.co.uk
250 2.1.5 Ok

Getting a 250 back from arbitrary mail from: and rcpt to: indicates vulnerability to open relay attacks. Unlike the original sendmail line, the initial From has been forged. That shouldn't be much trouble if the open relay function isn't there, which it isn't because rcpt to is the one and final destination, corresponding to the processing MTA.

Spoofing using multi-line sendmail

Another way to interact with the SMTP server:

anotheruser@vps1:~$ sendmail -v mailbox@silverbullets.co.uk
From: serioussam@example.com
To: jokers@example.com
Subject: Here is a subject
Here is a message body.
.

That works. From and To are not SMTP fields. Strictly speaking, they are message content. MAIL FROM and RCPT TO are the SMTP fields.

I tried:

anotheruser@vps1:~$ sendmail -v mailbox@silverbullets.co.uk
MAIL FROM: serioussam@example.com
RCPT TO: jokers@example.com
Subject: Here is a subject
Here is a message body.
.

That didn't work. It appears something had killed my sending, exim, MTA. I received non-TFO mode connection attempt to <<IP of target MX>>, 0 data. I got no more output. Further access to that server, even using previously successful commands such as telnet, curl, and ping, was temporarily suspended, with increasing severity.

I purged the undeliverable emails from the retry queue:

sudo exiqgrep -ix | xargs sudo exim -Mrm
sudo exiqgrep -iz | xargs sudo exim -Mrm

Fighting the ban

So seriously do people take these attempts at spoofing that I could no longer reliably access the silverbullets.co.uk host from my other VPS. Or my OpenVZ server has a duff exim installation.

I was able to test and re-establish connection a couple more times, but it involved a lot of trial, error, and waiting. Rebooting, restarting exim, and purging queued email all helped; to a point. My hostname reverts to the provider default (eg HostNamaste11821) on reboot, which took a while to spot and correct.

When I briefly re-gained the ability to send emails to silverbullets.co.uk it was short lived. I'd eventually be met by:

== mailbox@silverbullets.co.uk R=dnslookup T=remote_smtp defer (-54): retry time not reached for any host for 'silverbullets.co.uk'

For comparison, if I stop the postfix container listening to port 25 I'd receive this:

== mailbox@silverbullets.co.uk R=dnslookup T=remote_smtp defer (111): Connection refused

I don't know where exactly I'm prevented from accessing my server. That last observation about port 25 indicates the receiver still influences access, even if I can't see it in the receiver's logs. I still suspect cloud infrastructure and monitoring have detected me spoofing, or someone complained; I should have stuck religiously to my own email addresses. I'm not on Spamhaus's block list at least. It wasn't just port 25 that was blocked, but ping, bi-directionally, and curl too.

For the record, the sending MTA, though blocked no matter what I now do to it, does still email my GMX account with its Fail2Ban activity.

The only options left for this test runner are:

• waiting or
• reinstalling the whole server (since that is how I got exim on this OpenVZ VPS).

Let's explore some alternatives that don't risk impugning my VPS.

Testing with mxtoolbox.com

I've used mxtoolbox. Here is its session transcript:

Connecting to 107.175.0.0

220 83c72e74a80a ESMTP Postfix (Debian/GNU) [180 ms]
EHLO keeper-us-east-1d.mxtoolbox.com
250-83c72e74a80a
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250-SMTPUTF8
250 CHUNKING [113 ms]
MAIL FROM:<supertool@mxtoolboxsmtpdiag.com>
250 2.1.0 Ok [131 ms]
RCPT TO:<test@mxtoolboxsmtpdiag.com>
454 4.7.1 <test@mxtoolboxsmtpdiag.com>: Relay access denied [113 ms]

LookupServer 1465ms

Here is its assessment:

SMTP valid hostname should have matched. I've requested that it does, and two lines down the result says "OK Reverse DNS Mismatch".

SMTP banner check warning, again cites reverse DNS problems. I'm pretty sure that was all fixed to improve my reputation for sending emails. If I click the more info link to the right of these warnings, it takes me to mxtools.com's paid service.

I checked rDNS with dig -x 107.175.0.0. It was fine:

;; ANSWER SECTION:
18.245.175.107.in-addr.arpa. 3600 IN    PTR silverbullets.co.uk.

I can't argue with the times, or "Not an open relay", but I'm starting to question the integrity of this site.

"Supports TLS" is ambiguous. I know I support it when sending, but I haven't given it access to a certified key that should be needed to receive emails using encryption. The session transcript show that TLS support may have been questioned, but wasn't used. Compare: TLS validation: swaks enforcing tls snakeoil with TLS validation: swaks without ensuring tls. It's missing the TLS section.

What appears on the receiver

This is /var/log/mail.log:

Oct 28 19:57:43 silverbullets postfix/smtpd[186]: connect from keeper-us-east-1d.mxtoolbox.com[18.209.86.113]
Oct 28 19:57:44 silverbullets postfix/smtpd[186]: NOQUEUE: reject: RCPT from keeper-us-east-1d.mxtoolbox.com[18.209.86.113]: 454 4.7.1 <test@mxtoolboxsmtpdiag.com>: Relay access denied; from=<supertool@mxtoolboxsmtpdiag.com> to=<test@mxtoolboxsmtpdiag.com> proto=ESMTP helo=<keeper-us-east-1d.mxtoolbox.com>
Oct 28 19:57:44 silverbullets postfix/smtpd[186]: disconnect from keeper-us-east-1d.mxtoolbox.com[18.209.86.113] ehlo=1 mail=1 rcpt=0/1 quit=1 commands=3/4

That was the end of it, the transcript was complete, it disconnected after failing the open relay attempt. Very elegant.

Testing with microsoft.com

testconnectivity.microsoft.com contains tools for testing inbound and outbound emails. It actually emailed me, seen here in the mail log:

Oct 28 21:10:01 silverbullets postfix/smtpd[220]: disconnect from smtp.testexchangeconnectivity.com[52.109.8.10] ehlo=1 mail=2 rcpt=1/2 data=1 commands=5/6

The website's transcript demonstrated it "used" STARTTLS, in the same sense that mxtoolbox.com "used" STARTTLS:

EHLO Response 250-83c72e74a80a
250-PIPELINING
250-SIZE 10240000
250-VRFY
250-ETRN
250-STARTTLS
250-ENHANCEDSTATUSCODES
250-8BITMIME
250-DSN
250-SMTPUTF8
250 CHUNKING

It tested a lot of stuff and provided a lot of green ticks. Microsoft agreeing with mxtoolbox offers peace of mind. Is this a false confidence?

On with the tests...

Testing with an open source tool

SMTPTeste is a neglected hobby project with a good few stars and the stated objective to test for 3 common vulnerabilities. It introduces me to smtplib which is another, batteries-included, python library to help me test.

For now, I can be pretty sure this isn't going to cover anything that wasn't already tested, but it's nice to escape dependency on closed sourced third parties.

Testing from home

Locally, I built and ran the same postfix container I am using to receive email in the cloud. I was able to send an email again, using telnet, because that seemed the safest agent before.

What I received was exactly analogous (I spoofed gmx.com instead of example.com) to what I received when sending from my other VPS.

The same two defects remained:

1. Received: shows my home broadband DNS.
2. Authentication-Results. dkim=permerror (bad message/signature format).

Picking up from my now blocked VPS, at home

This, single-line sendmail, I believe, is what led to my loss of connectivity between cloud VMs:

echo -e "MAIL FROM: genuine_own_user@gmx.com\nRCPT TO: mailbox@silverbullets.co.uk\nFrom: genuine_own_user@gmx.com\nSubject: danger zone\nEso es muy peligroso.\n" | sendmail -v mailbox@silverbullets.co.uk

I needed to correct /etc/mailname from the default example.com (that domain really messes with email) for this to arrive on my receiving MTA.

In the received email From, From:, and Return-Path all used /etc/mailname, gmx.com. My explicit specifying of MAIL FROM: and RCPT TO: only created extra, non-SMTP, fields in the message body. Not even Subject: appeared believable, and was absent from the receiver's mail interface.

postfix logs

/var/log/mail.log references undelivered emails, potentially being retried every minute.

/var/log/mail.err and /var/log/mail.warn are full of expected DKIM errors. My dockerfile and image use a fictitious private key. /var/log/mail.info elaborates.

/var/log/mail.log on postfix during telnet

Here is the communication flow between a telnet client and postfix server:

TLS qualms

I put a lot of time into automating TLS for my website. It doesn't happen by accident.

No third party is complaining, but I can't prove a negative. This post is long enough, and appears to show everything is fine without hardened TLS. TLS discussion can be found here.

To summarise that post: Mail servers and testers appear lenient in accepting self-signed certificates. It's better than nothing, but clients have no guarantee that the certificate we supply is genuine, opening them up to man in the middle attacks. Except it's not quite that simple since we are signing with DKIM, which is verifiable from our DNS server. I don't know the details of whether the initial supply of the TLS certificate is signed with DKIM, who goes first?

This post focuses on resilience in the sense of not getting pwned and not suffering reputational damage. TLS is more about ensuring privacy of email communication. Recognised TLS certificates aren't difficult to provide, so it's worth reading my post on TLS. Previously, I have observed exim report:

Warning: No server certificate defined; will use a selfsigned one.

Suggested action: either install a certificate or change tls_advertise_hosts option

I don't have the context of what caused this, but, taken at face value, it is clear that exim attaches little value to self signed certificates and is able to pull one out as a contingency if it doesn't have a trusted, signed, one.

That mail interface

For interfacing users with mail, we have IMAP and POP3 services. These allow third party mail clients to communicate securely with the SMTP server. Security first, cosmetics and user-friendliness second.

These IMAP services are popular with postfix:

• dovecot
• courier-imap

Conclusion

If we need to spoof email, telnet is our friend. It has proved more reliable and available than MTAs. The only drawback is we need some understanding of the SMTP protocol. At the other end of the scale, third parties offer their expertise for free, but can leave us overly dependent on them.

I'd always told myself I'd use the vulnerability scanners I was trained in. Now they appear more about (re)learning the tool than the potential vulnerabilities and how to fix them.

We saw exim, at least in the cloud, fall by the wayside as we upped the evilness of our tests. Privately, on a number of occasions, I've seen postfix retort something along the lines of, "if you don't play by the rules, I won't either", but I couldn't recreate that by choice.

Received: from could never be faked. This will be good to know (and assert) if we should need a further hand in inward mail filtering.

Return-Path, on the other hand could easily be faked, see Spoofing using telnet.

MAIL FROM was not explicitly mentioned in the SMTP header. It is represented by that first "From " line.

RCPT TO doesn't matter to me at this point, the email will go where it is sent. Open relay attacks need my server to accept email not destined for its domain, which my server does not do.

Janet, formerly the joint academic network, summarises acceptable relaying as:

• internal IP address to internal mail address, permitted.
• internal IP address to external, permitted.
• external to internal mail address, permitted.
• external to external, FORBIDDEN.

The last being the open relay function. postfix logs contain relay fields, but they are localhost or the destination MX server.

From my correspondence with Outlook.com, they want me either to make my case again, or register for one of their services that notifies about volumes of junk mail I am sending. I think they must lump me with other senders in a /24 subnet. To register I must have incoming mail.

This post, and that one on TLS prove my postfix installation secure. To keep it up, inviting email, as a going concern, I'm going to need to look at Spam Assassin.

Search

Categories

Tags