SMTP Security via Opportunistic DANE TLS

3494 words (14 pages) Essay

19th Apr 2018 Computer Science Reference this


Disclaimer: This work has been submitted by a university student. This is not an example of the work produced by our Essay Writing Service. You can view samples of our professional work here.

Any opinions, findings, conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of




This paper provides an insight about a protocol designed for SMTP transport, which offers downgrade resistance. This protocol deploys as security for Domain Name System (DNS) Authentication of Named Entities (DANE) based Mail Transfer Agents (MTA). To a client who uses authenticated and encrypted transport layer security (TLS), using this protocol provides a support to the Internet email and allows incremental transition. This paper talks about a new connection security model for Message Transfer Agents (MTAs). Message Transfer Agents are responsible for the transfer of electronic mails to other computers. This connection model is based on a fact that the received end server is chosen indirectly using DNS (Domain Name System) Mail Exchange (MX) Records.

This paper talks about the SMTP channel security and analyses why the current security model is inefficient and the need for a new model to protect Simple Mail Transfer Protocol (SMTP) traffic. This paper elaborates on various other aspects such as Mandatory TLS security and DANE authentication and various operation considerations that are required are discussed.

1. Introduction

Simple Main Transfer Protocol (SMTP) states a new connection security model for Message Transfer Agents (MTAs). Key features of inter-domain SMTP delivery inspire this model; in precise the fact that the endpoint server is selected indirectly via Domain Name System (DNS) Mail Exchange (MX) records and that with Mail Transfer Agent (MTA) to Mail Transfer Agent (MTA) SMTP the use of Transfer Layer Security (TLS) is generally opportunistic.


  1. SMTP Channel Security

With HTTPS, Transport Layer Security (TLS) engages X.509 certificates allotted by one of the various Certificate Authorities (CAs) hustled with famous web browsers to allow users to authenticate their “secure” websites. Before we specify a new DANE TLS security model for SMTP, we will explain why a new security model is needed. In the procedure, we will discuss why the similar HTTPS security model is insufficient to protect inter-domain SMTP traffic.

The sections described below are the four main key problems with applying the traditional PKI to SMTP protocol that is tackled by this specification. Since the SMTP channel security approach is not clearly explained in either the receiver address or Mail Exchange (MX) record, a new signaling mechanism is wanted to specify when channel security is possible and should be used. The publication of the Transport Layer Security (TLSA) records will permit the server operators safely signal to the SMTP clients that the TLS is available and must be used. DANE TLSA makes this possible to simultaneously determine which endpoint domains support the secure delivery via TLS and tells how to verify authenticity of associated SMTP services, affording a path forward to the ubiquitous SMTP channel security.

  1. STARTTLS downgrade attack

The Simple Mail Transfer Protocol (SMTP) is a single hop protocol in multi hop store and forward the email delivery procedure. SMTP envelope recipient addresses are not transport addresses and safety agnostic. Unlike Hypertext Transfer Protocol (HTTP) and its related secured versions, HTTPS, where the use of (TLS) Transport Layer Security is signaled via URI scheme, transport security policies are not directly signaled by the email recipients. Certainly, no such signaling can work well with this SMTP since TLS encryption of SMTP protects the email traffic on a hop by hop basis while email address can only express end to end policy.

SMTP relays employ best-effort “opportunistic” security model for TLS with no mechanism existing to signal transport security policy. A single SMTP server TCP listening end point can serve both the TLS and the non-TLS clients; the use of this TLS is negotiated via SMTP STARTTLS command. The client is supported by the server signals TLS over a clear text SMTP connection, and, if client also supports the TLS, it may negotiate TLS encrypted channel to use for the email transmission. An MITM attacker can easily suppress the server’s indication of TLS support. Thus pre DANE SMTP TLS security can subvert by simply decrease a connection to clear text. The TLS security features, such as use of PKIX, can stop this. The invader can simply disable the TLS.

  1. Insecure server name without DNSSEC

The records abstract next hop transport end point with the SMTP DNS (MX) Mail Exchange and allow the administrator to specify set of targets server to which SMTP traffic should be pointed for given domain.

Until and unless PLIX TLS client is vulnerable to MITM attacks it verifies that servers certificate binds public key to a name that match one of the clients reference identifiers. Servers domain name is the natural choice of reference identifiers. However, server names are taken indirectly via Mail Exchange records with SMTP. The Mail Exchange lookup is susceptible to MITM and DNS cache destroying attacks without DNSSEC. Active attackers can forge the DNS replies with the fake mail exchange records and can send email to servers with the names of their selection. Therefore, secure proof of SMTP TLS certificates matching server name is not conceivable without the DNSSEC.

One could try to harden the TLS for the SMTP against the DNS attacks by using envelope receiver domain as a situation identifier and needful each SMTP server to have a confidential certificate for envelope recipient domain rather than mail exchange hostname. Unfortunately, this is impractical as third parties that are not in a position to obtain certificates for all the domains they serve handle email for many domains. Deployment of (SNI) Server Name Indication extension to TLS is no cure, since SNI key organization is operationally inspiring except when email service provider is also fields registrar and its certificate issuer; this is hardly the case for email.

Since recipient domain name cannot be used as SMTP server orientation identifier, and neither can the mail exchange hostname without the DNSSEC, large scale deployment of authenticated transport layer security for SMTP needs that the DNS be safe.

Since SMTP protocol security depends on DNSSEC, it is important to point out that consequently SMTP with the DANE is most traditional possible trust model. It trusts only what must be important and no more. Adding any other trusted actors to the mixture can only reduce the SMTP security. A sender might select to more harden DNSSEC for selected high value getting domains, by organizing explicit hope anchors for those domains in its place of relying on the chain of trust from root domain.

  1. Sender policy does not scale

Transfer systems are in some cases openly configured to use the TLS for mail directed to designated peer domains. This needs sending MTAs to be organized with appropriate subject names digests to expect in accessible server certificates. Because of heavy administrative burden, such statically configured SMTP secure channels are used rarely. Internet email, on other hand, requires regularly contacting new domains for which the security configurations cannot be reputable in advance.

Abstraction of SMTP transport end point via DNS MX records, often across society boundaries, limits the use of the public CA PKI with the SMTP to a minor set of sender configured peer domains. With the little opportunity to use the TLS authentication, transfer MTAs are rarely configured with a complete list of trusted CAs. SMTP services that support STARTTLS often deploy X.509 certificates that are self signed or distributed by a private CA.

  1. Identifying applicable TLSA records
    1. DNS considerations
      1. DNS errors, bogus and indeterminate responses

SMTP client that implements the opportunistic DANE TLS per specification depends on the integrity of the DNSSEC lookups. This section lists DNS resolver requirements needed to avoid the downgrade attacks when using the opportunistic DANE TLS.

SMTP clients shadowing this specification SHOULD NOT distinguishes between “insecure” and “indeterminate”. Both “insecure” and “indeterminate” are handled identically in either case invalidated data for query domain is all that is and can be available, and verification using data is impossible. In what shadows, when we say the word “insecure”, we also include DNS results for the domains that lie in portion of DNS tree for which there is no appropriate trust anchor. With the DNS origin zone signed, we expect that validating resolvers used by Internet facing MTAs will be configured with the trust anchor data for root zone. Therefore, “indeterminate” domains should be rare in rehearsal.

A security aware DNS resolver MUST be able to determine whether given non error DNS response is “secure”, “insecure”, “bogus” or “indeterminate”. It is expected that the most security alert stub resolvers will not signal an “indeterminate” security position in the application, and will sign a “bogus” or error result instead. If a resolver does signal an “indeterminate” security status, SMTP client MUST treat this as a “bogus” or error result had been returned.

  1. DNS error handling

When an error or “bogus” or “indeterminate” prevents an SMTP client from defining which SMTP server it should attach to, message delivery MUST be late. This naturally includes, for example, the situation when a “bogus” or “indeterminate” response is faced during MX resolution. When several MX hostnames are obtained from popular MX lookup, but a later DNS lookup failure stops network address determination for a given MX hostname, delivery may continue via any lasting MX hosts.

When the particular SMTP server is firmly identified as delivery destination, a set of DNS lookups must be done to find any linked TLSA records. If any DNS queries used to trace TLSA records fail, then SMTP client must treat that server as inaccessible and MUST NOT deliver messages via that server. If no servers are nearby, delivery is delayed.

  1. Stub resolver considerations

A note about domain name aliases, a query for domain name whose ancestor domain is a DNAME alias returns the DNAME RR for ancestor domain, along with the CNAME that maps query domain to the consistent sub domain of target domain of domain name aliases. Therefore, when we speak of CNAME aliases, we indirectly allow for the likelihood that alias in question is the result of ancestor domain DNAME record. Therefore, no explicit support for DNAME records is wanted in SMTP software, it is enough to process resulting CNAME aliases. DNAME records require special processing in validating stub resolver library that checks integrity of the joint DNAME plus CNAME. When a local caching resolver, rather than the MTA itself handles DNSSEC validation, even that part of DNAME support logic is outside MTA.

  1. TLS discovery

Opportunistic TLS with the SMTP servers that advertise TLS care via STARTTLS is topic to an MITM downgrade attack. Also some of the SMTP servers that are not, in fact, the TLS capable mistakenly advertise STARTTLS by evasion and clients need to be ready to retry clear text sending after STARTTLS fails. In contrast, the DNSSEC legalized TLSA records must not be published for the servers that do not support the TLS. Clients can safely understand their existence as a promise by the server operative to implement the TLS and STARTTLS.

SMTP client may organize to require DANE verified delivery for some destinations. We will call such a configuration as “mandatory DANE TLS”. With mandatory DANE TLS, distribution proceeds when “secure” TLSA report are used to establish an encrypted and authenticated TLS channel with SMTP server.

  1. MX resolution

In this we consider next hop domains, which are subject to MX resolution and also have MX records. TLSA records and its associated base domain are derived disjointedly for each MX hostname that is used to effort message distribution. DANE TLS can validate message delivery to intend next hop domain only when MX records are obtained firmly via a DNSSEC validated lookup.

MX records must sort by preference; MX hostname with worse MX preference that has TLSA records MUST NOT preempt MX hostname with better preference that has no TLSA records. In other words, stoppage of delivery loops by following MX preferences must take priority over channel safety considerations. Even with 2 equal preference MX records, MTA is not obligated to choose MX hostname that provides more security. Domains that need secure inbound mail delivery have to ensure that all of their SMTP servers and their MX records are organized accordingly.

  1. Non-MX destinations

It describes algorithm used to locate TLSA records and related TLSA base domain for an input domain not subject to MX resolution. Such domains include: Each (MX) mail exchange hostname used in message delivery attempt for an original next hop endpoint domain subject to the MX resolution. Any superintendent configured relay hostname not related to MX resolution. This often involves configuration set by MTA administrator to handle some mail. Next hop target domain subject to MX resolution that has no MX records. In this case domains name is implicitly and also its sole SMTP server name.

  1. TLSA record lookup

Each candidate TLSA based domain is in turn prefaced with service labels of form “_._tcp”. Resulting domain name is used to release a DNSSEC query with query type set to TLSA.

For SMTP, destination TCP port is usually 25, but this may be dissimilar with the custom routes stated by MTA administrator in which case SMTP client MUST use appropriate number in the “_” prefix in place of “_25”. For example, candidate based domain is “”, and SMTP connection is to port 25, TLSA RRset is gained via DNSSEC query of form:

  1. DANE authentications

It describes which TLSA records are appropriate to SMTP opportunistic DANE TLS and how to use such records to authenticate SMTP server. With opportunistic DANE TLS, both TLS support implied by the occurrence of DANE TLSA records and verification parameters needed to authenticate TLS peer are found composed. In contrast to protocols where exclusively the client sets channel security policy, authentication via this protocol is predictable to be less prone to linking failure caused by mismatched configuration of client and the server.

  1. TLSA certificate usage

The DANE TLSA defines number of TLSA RR types via mixtures of three numeric parameters. Rest of TLSA record is “certificate association data field”, which stipulates full value of a certificate. The parameters are: TLSA Certificate Usage field, the selector field and matching type field.

  1. Certificate usage DANE-EE (3)

Authentication via certificate usage DANE-EE (3) TLSA records includes simply checking that server’s leaf certificate equals the TLSA record. In particular, the requisite of server public key to its name is built on the TLSA record association. The server MUST be considered authenticated even if none of the names in certificate matches client’s reference identity for the server.

Similarly, expiration date of server certificate MUST be ignored; validity period of TSA record key requisite is dogged by validity pause of the TLSA record DNSSEC signature.

  1. Certificate usage DANE-TA (2)

Some domains might prefer to evade operational complexity of publishing the unique TLSA RRs for each and every TLS service. If domain employs a mutual issuing Certification Authority to create the certificates for the multiple TLS services, it might be simple to publish issuing authority as the trust anchor (TA) for certificate chains of all the relevant services. TLSA query domain for each and every service issued by same TA may then be set to CNAME alias that points to common TLSA RRset that matches TA. For example: IN MX 0 IN MX 0 IN CNAME IN CNAME IN TLSA 2 1 1 e3b0c44298fc1c14….

  1. Certificate usage PKIX-TA (0) and PKIX-EE (1)

SMTP clients cannot, without trusting on DNSSEC for safe mail exchange records and the DANE for STARTTLS care signaling, perform server identity proof or stop STARTTLS downgrade attacks. Use of PKIX CAs offers no additional security since an attacker is capable of compromising the DNSSEC is free to replace any of PKIX-TA (0) or the PKIX-EE (1) TLSA records with the records bearing any suitable non PKIX certificate usage.

  1. Certificate Matching

SMTP client must use the TLSA records to authenticate SMTP server when at least one usable “secure” TLSA is found. Through SMTP server messages must not be delivered if the authentication fails, otherwise SMTP client is vulnerable to the MITM attacks.

  1. Server key management

Before employing the new EE or TA certificate or public key, two TLSA records MUST be published, one matching currently deployed key and other matching new key is scheduled to replace it. Once a sufficient time has elapsed for all the DNS caches to expire previous TLSA RRset and related signature RRsets, servers may be organized to use new EE private key and associated public key certificate or may service certificates signed by a new trust anchor.

Once the new public certificate is in use, TLSA RR that matches retired key can be removed from the DNS, leaving only RRs that matches certificates in energetic use.

  1. Digest algorithm agility

The DANE TLSA specifies a many number of digest algorithms; it does not specify protocol by which SMTP clients and TLSA record publishers can agree on strongest shared algorithm. Such a protocol will allow client and the server to avoid exposure to any deprecated weaker algorithm that are published for the compatibility with fewer capable clients, but should be unnoticed when possible.

  1. Mandatory TLS security

MTA implementing this protocol might require a stronger safety assurance when sending e-mail to the selected destinations. Sending organization may need to send the sensitive e-mail or may have regulatory responsibilities to keep its content. This protocol is not in the conflict with such a necessity, and in fact it can often simplify genuine delivery to such targets.

Specifically, with the domains that publish a DANE TLSA records for their mail exchange hostnames, a sending MTA can be arranged to use receiving domains DANE TLSA records to authenticate consistent SMTP server. Authentication via the DANE TLSA records is simpler to manage, as changes in receivers expected certificate things are made on the receivers end and don’t need manually conversed configuration changes. With the mandatory DANE TLS, when the no usable TLSA records are found, message distribution is delayed. Thus, the mail is only sent only when an authenticated TLS channel is recognized to the remote SMTP server.

Cite This Work

To export a reference to this article please select a referencing stye below:

Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.
Reference Copied to Clipboard.

Related Services

View all

DMCA / Removal Request

If you are the original writer of this essay and no longer wish to have your work published on the website then please:

Related Lectures

Study for free with our range of university lectures!