The story so far
Securing HTTP has luckily become a commodity over the last couple of years, even for mainstream sites which only did enable secure connections for authentication processes before, if at all. Browsers start to implement more features (HPKP, CSP, HSTS…) with even better usability and start to get rid of legacy protocols and ciphers. At this corner of the internet, the state of TLS looks really good and everyone feels great about green lock-signs. But whats about less obvious areas of secure communication?
E-Mail for example has seen lots of good initiatives to enhance security primarily with the goal of reducing Spam (DKIM, Greylisting, Domain validation…). Apart from that the situation still looks quite ugly. There are indeed options for skilled end-users to do end-to-end encryption using SMIME, PGP for mail clients and to use browser-based crypto or server-side solutions for webmail applications. However, these solutions are either too complicated or only cover niches. Ultimately users could do something to improve their security when sending mail, but there is no solution to demand receiving mail securely - while incoming mail represents the vast majority of John Doe’s mail traffic.
By design E-Mail - more specifically SMTP - depends on servers talking to each other. Clients (like Outlook, GMail or Apple Mail) submit their messages to the configured server and let SMTP figure out a suitable route to deliver the mail across the internet and local networks. This leads to a quite robust infrastructure but takes away influence on security if the user did not encrypt the message. There is no way to tell or define what route a mail will take and if those servers communicate in a secure way with each other. Even when running a well configured mail server yourselves there is always a chance that a incoming mail was transferred in plain-text between two other servers. The recipient can only figure this out after receiving the mail, which might have been compromised by then. Keeping in mind what kind of sensitive data gets sent around multiple billion times per day, this is really troubling.
Security vs. cost
Operators of larger mail services usually enable strong security features but the majority of smaller operators does not care as much. With an almost bullet-proof configuration, there is a huge chance of not being able to communicate to many services. For example, while GMail or Outlook.com may use top notch security, many mails are sent by non-public mail operators such like your bank, random businesses or small online-shops. These guys often have no clue about proper SMTP configuration or have to depend on ancient mail system configurations for the sake of compatibility. Having a substantial error rate, say 5%, when sending or receiving invoices, general purpose or business communication leads to high cost.
With this in mind, most operators chose to stick with a less secure but more compatible configuration. For the more security focused operators, this means they can’t remove less secure configuration, a classical chicken-and-egg problem. There have been some regional projects to raise security (e.g. “E-Mail made in Germany”) which however appear to exist for marketing purposes instead of establishing real security. The ridiculous name aside, this initiative just makes sure that the biggest consumer mail operators in a given country talk TLS to each other. Those popular operators had TLS for incoming and outgoing SMTP enabled anyway and such campaigns avoid to mention that the other 99% of operators don’t know or care.
Mails sent by humans via huge mail services are actually a minority, most are generated by automated systems. If your local lawyers office or hotel booking system sends data to GMail without encryption it does not matter that GMail and Yahoo talk securely with each other. Those major services simply don’t force encryption on incoming mails to become a commodity, instead they implement a good-enough configuration to avoid being blamed and seem to hope the situation will enhance by itself. This led to the obscene situation that nowadays cat pictures on the web are better secured than critical information transferred via mail.
Doing the test
Lets get into some practical examples. I chose to enable a secure incoming SMTP configuration only allowing commonly used ECDHE ciphers (which are expected to be “unbreakable” in the forseeable future) for my own mail server for a month and checked what kind of issues would happen. “Funny” enough, most practical issues affected financial institutions and large corporations. My assumption is that those guys have a tight corset of policies in place that suppress innovation or even keeping up with reality when it comes to secure configuration. Some examples of failed incoming delivery:
- Deutsche Post (DHL) - mx1.dhl.iphmx.com
- Vodafone (MMS service) - vgmmsyvr.mmslegacy.vodafone.de
- Paypal - mx1.slc.paypal.com
- Apple - msbadger0201.apple.com
- American Express - extmta1-new.aexp.com
In most cases the worst thing that happens is that mail does not get delivered. American Express (“AMEX”) is a little different though. The topic has been discussed with one of their “Senior Security Analyst Cyber Security Investigations” (so much for title porn) but did not lead to any real change. I guess two months after disclosing the issue to American Express and not getting any more response it’s fair to get the word out.
The AMEX situation
What AMEX like most financial institutions does is checking for geo-location, IP and other fraud relevant metadata when accessing a account. If someone successfully logs in from France and just 10 minutes later from Japan thats reasonable suspicious to block access. Good idea. They then send a some-digit “authentication code” to the account holders mail address as additional factor to authorize the login. The same happens quite often when accessing the same account simultaniously from mobile and stationary internet connections since the IP and geo-location information changes, which gave me the opportunity to do research without traveling.
When the authentication code is delivered, AMEX does not handle ECDHE ciphers for SMTP and delivery fails.
Dec 23 16:01:05 box postfix/smtpd[25349]: SSL_accept error from extmta1-new.aexp.com[148.173.96.86]: -1
Dec 23 16:01:05 box postfix/smtpd[25349]: warning: TLS library problem: error:1408A0C1:SSL routines:SSL3_GET_CLIENT_HELLO:no shared cipher:s3_srvr.c:1440:
OK, so no authentication code for me, sad. But wait!
Dec 23 16:29:16 box postfix/smtpd[30013]: connect from extmta1-new.aexp.com[148.173.96.86]
Dec 23 16:29:17 box postfix/smtpd[30013]: 1FBCDAE88B: client=extmta1-new.aexp.com[148.173.96.86]
Dec 23 16:29:17 box postfix/cleanup[30017]: 1FBCDAE88B: message-id=<UMS255338020161223042913.DEUDEUMSPWR0001.ENG-ALERTS@welcome.aexp.com>
Some minutes later the same MTA connects again and succeeds. So what happened? Did they fixed their MTA in the meantime and trigger a retry? Looking at the received mails headers suggests otherwise…
# Received: from welcome.aexp.com (extmta1-new.aexp.com [148.173.96.86])
# by mx.heiland.io (Postfix) with ESMTP id 1FBCDAE88B
# for <recipient@replac.ed>; Fri, 23 Dec 2016 16:29:16 +0100 (CET)
The absence of TLS information at the header indicates that AMEX indeed has a fall-back for failed mail delivery - but one which drops any kind of encryption. This means my second factor to validate legitimate access to my credit-card account just got sent through the internet as plain-text since outgoing AMEX MTAs are incapable of handling 2015-ish security configurations. Ugh.
So what could be done with that? At the very least it makes passive attacks and sniffing shared networks a viable option to acquire this secure token. Some attacker could look out for reasonably secure mail servers to which AMEX is incompatible to and look out for re-tried unencrypted mail without launching an active man-in-the-middle attack. This might be a hard thing to pull off between AMEX and GMail, but certainly when thinking about smaller corporate mail services. Legitimate recipients may not notice the issue since they still get their authentication code, just a bit later. AMEX is quite huge in the corporate credit-card business so there is a good chance to hit some high-profile accounts with interesting data. I did not get into more detail with this but there certainly are some scenarios where this issue degrades passive security for the authentication code to near-zero. AMEX is just the example that came to my attention, there is no reason to believe this would be a unique case.
Lessons leaned
So what are my results from this one-month experiment?
- Using current generation encryption for SMTP leads to failed deliveries.
- It does not only affect small but also large companies. Public mail providers were unaffected.
- Forcing high security standards may even lead to drastically reduced security.
Folks, please fix your MTAs.