Email Will Never Die
Take a look at any blog related to email and there will inevitably be a blog post proclaiming that Email Is Not Dead. There are literally billions and billions of results for the term on Google.
One look at the eponymous emailisnotdead.com and you’ll learn that over four billion people have an email address and that nearly one million email messages per second are passing around the internet. A single platform sent nearly 6 billion emails on Cyber Monday alone, and that was a fraction of the total email sent that day.
Email continues to thrive because it is decentralized, standards-based, outside the control of any single entity, and has a 100% adoption rate.
That said, unlike certain undead creatures, email is also not stagnating. Email is fundamentally some headers and content wrapped in a digital envelope passed from server to server, but each piece of the email ecosystem continues to innovate and evolve.
The Move to the Cloud
One of the more recent trends in the email ecosystem has been the move of email infrastructure to the cloud.
While there have been Email Service Providers (or ESPs) for some time, historically the focus has been on bulk email: managing lists, content, and scheduling on behalf of the sender as a cohesive tool. These were Software as a Service (SAAS): a suite meant to handle all aspects of your email marketing. Many of these ESPs would provide services around transactional messages and other message relaying, but it wasn’t the focus of their offering.
One new development was the rise of the email relay service providers: companies that didn’t focus on the creation of messages, but instead on the relaying of those messages to their destination. These Infrastructure as a Service (IAAS) companies took care of the difficult tasks of scaling email infrastructure on behalf of the sender and helped manage the deliverability challenges those senders would otherwise face.
The companies focus primarily on the relaying of transactional mail: messages that are typically generated as the result of a user action and include messages such as password resets, signup confirmations, purchase receipts, activity notifications, and status updates. Typically these companies don’t provide tools for things like list management, content management, or schedule management, but will often provide APIs that make it easy for developers to submit a template, recipients, and substitution data for the platform to merge together into a batch of messages.
The Appeal of the Cloud
There has been rapid growth over the last few years in the number of cloud email relay vendors, including offerings from companies of all sizes including:
- Amazon Simple Email Service (SES)
Why the rise in popularity?
It has a lot to do with the increasing complexity of delivering mail to the inbox.
It used to be fairly straightforward to send email to your users: the server your application was hosted on likely already had a built-in email server, and you simply assembled a basic message in your code, pointed it at the server, and hit send. Within a few seconds the message appeared in your inbox and you moved on to building the next part of your killer app. Even today, most developers will send a couple of quick tests, see them arrive in the inbox, and consider their app ready for prime time.
Unfortunately it all too often only works in testing; go from sending a couple of messages a day to even a couple of messages per hour and the issues start to appear. First, maybe the app slows down because the code to generate and send the messages wasn’t built to scale. Next, the messages stop reaching the inbox because the mailbox providers filtered the message because of non-compliance.
The bottom line is that there are many skills required to send real volumes of commercial email and have them reach the inbox, divided into multiple disciplines that are often handled by multiple people. Commercial email truly follows Bushnell’s Law as something that is “easy to learn, difficult to master”.
For any small businesses or startups looking to send email as part of their service (and at this point are there really any apps and services that don’t have an email component?) it certainly makes sense to start with a cloud provider where all they have to do is call an API with the recipient data and a template ID and leave the rest up to the vendor.
That said, it’s not just startups that use cloud email services. Many large and mature companies are also using IAAS vendors to take care of their sending needs for reasons that include having a cloud focus, or an awareness of the skills required to be successful and the relative scarcity of top email professionals to manage their infrastructure and deliverability.
Not Every Cloud has a Silver Lining
If things are so great in the cloud, why is anyone still sending on-prem? Let’s review some of the common reasons that organizations of various reasons are still sending on-prem:
If it ain’t broke, don’t fix it.
There are systems in place that are older than some of our readers, and they have been sending emails for decades without issue. These systems typically avoid the deliverability issues faced by new senders because of their long history and reputation with mailbox providers, with the only issue they face potentially being one of scalability. Systems like this are unlikely to have new scalability challenges because their growth phase is typically already passed.
That’s not to say there wouldn’t be benefits to moving legacy systems to the cloud, but depending on how these systems were built, the move could be complicated enough that the costs don’t outweigh the benefits of the change. Legacy systems are often hard-coded in such a way that changing them to inject to an external host is difficult enough without then having to integrate the event data from the external system.
One common objection I encounter to moving messaging to a cloud provider comes from regulated industries: their security policies simply do not permit the use of third-party cloud services.
Most banks, insurance companies, and HIPAA-compliant medical firms have security policies that can be very strict regarding the use of third-party platforms that touch their data. While it could be argued that anything being sent in an email should not contain Personally-Identifying Information (PII) in the first place, and therefore whatever you send to a cloud email relay vendor is free from PII, some of these organizations consider even the logs containing the recipient email address to be PII and therefore ineligible for third-party cloud hosting.
Cloud email relay vendors take care of deliverability for you, and that includes taking care of sender reputation. That reputation is judged across multiple datapoints by the mailbox providers including sending domain, sending IP, and even the range of sending IPs used by the organization. Some of their customers are on dedicated IPs, and some are on shared IPs. Even those sending on dedicated IPs can potentially impact the reputation of other customers in certain circumstances.
Because they have to look out for the reputation of all their customers, these vendors have compliance teams that work to ensure that all traffic on their system presents a minimal risk to sending reputation.
There are a number of potential users for these services that send legitimate mail, but will be judged as too risky by the compliance teams at these various vendors. This can include opt-in subscriptions to companies such as casinos, cannabis companies in legalized jurisdictions, lender marketing companies, and other categories that are perfectly legal and legitimate, but deemed by the vendor as too potentially risky for their purposes.
Senders who fall into these categories will often find themselves having to send their own mail from their own servers but will have to be especially diligent in their practices to ensure that they put their best foot forward to make a good first impression with the mailbox providers.
Insights and Control
Because the cloud email relay vendors are targeting users who are not email experts, they tend to produce products that are more of a “black box” to their users: the user injects mail into the platform, the mail comes out the other end, and the user receives some reporting as to how many messages delivered or failed, with some basic data on why the failures occurred. In addition, many vendors provide tracking for engagement data, showing how many recipients opened messages and clicked on links.
For many users, this is sufficient. The goal is to get the message out and get it read by the user. It’s nice to know those things happened, and there’s no additional need for introspection. When things don’t go as planned, these vendors provide support and often additional deliverability teams to help identify and correct issues that arise. This is exactly what the average user of a cloud email relay vendor wants, just to submit the message and leave the rest to the vendor.
While that works for many smaller senders, larger and more sophisticated senders tend to reject the “black box” approach: they are accustomed to having extensive and direct insight into their sending environment and expect highly granular detailed information about everything in their message flow.
In addition to insight, these organizations are accustomed to control and access to their systems. They want the ability to modify and adjust their environment, and direct access to resolve issues as they appear. These teams don’t want to submit a ticket and wait, they are used to digging in and resolving issues they encounter in a timely manner.
Another common objection I’ve encountered from on-prem senders when presented with a third-party cloud option concerns a lack of customization and policy options. When you use a cloud service, what you inject is pretty much what gets sent, and for most senders, that is what is expected.
What the typical small sender doesn’t realize is that such an approach puts the onus on them to ensure that the message they inject is exactly what they want sent. When the total sending environment consists of one or two systems that create messages for relay that is pretty simple, but larger environments can face a number of challenges with that scenario.
Imagine you are a large enterprise with hundreds of systems that create email messages, and your deliverability team learns that going forward, Gmail would like you to add a Precedence: Bulk header to all your bulk messages going out of your environment. The amount of work required to go into each and every one of those systems and modify the message generation code would not only be monumental, but it may also be nearly impossible if you don’t have access to the source code or developer knowledge involved in creating some of those systems.
What if you had the ability to catch those messages mid-stream, evaluate whether the message came from a source that generates bulk messages, and add the header to those messages in transit? That’s exactly what several of the on-prem email servers are capable of. These Message Transfer Agents, or MTAs, often have scripting capabilities that make it a simple matter to adjust and manipulate not only message headers, but their content and sending patterns dynamically. Whether you love them or hate them, another example of this capability is the legal notices that often get appended to outgoing corporate email messages.
Many of the on-prem senders I work with have dozens and even hundreds of automated steps that their messages pass through before they get relayed to the outside world, ensuring that those messages are compliant, clean, and properly formatted regardless of which of the dozens of internal and even external systems generated the message.
For a startup or small sender, using a cloud email provider is a no-brainer; if you’re sending a few hundred or even a couple of thousand email messages a month, the cost is literally zero. There are multiple vendors who offer free and developer accounts for small senders and make it very easy to either start a new service or migrate a small one with little expense and effort.
As a company grows, the economics of scale stay with the use of a cloud email relay vendor for volumes that can get into millions of messages per month.
As these companies scale up their sending, they start to look at the different costs associated with hiring skilled MTA administrators, deliverability experts, network engineers, and the other various professionals required to successfully send email on-prem. Because cloud vendors charge a rate that translates into a cost for every email sent, there is always a tipping point where it is more economical to make the capital investment in building an on-prem sending environment compared to the ongoing operational cost in paying per message to depend on a cloud email relay vendor.
Of course, if an organization is already sending on-prem, the math is even more conducive to staying on-prem: the capital expense of the MTA is already spent, the maintenance on software and hardware is in place, and the staff with the relevant skills are already present. It is rare, but not unheard of, for a large-scale sender to make the switch to a third-party cloud vendor for their sending needs.
Even those large organizations looking to move to the cloud are most likely looking to move to a private cloud solution: virtualizing their infrastructure into either their own datacenter on top of a virtualization solution such as VMware, or into a public cloud provider such as Azure or AWS, while still retaining control of the software they deploy.
Companies that move to a private cloud get the benefits of management, redundancy, and scalability that the cloud offers, while retaining the control and insights they enjoyed in their previous on-prem deployment.
Companies that look to move to a public cloud platform have to deal with migration challenges around things like IP reputation, IP warmup, and infrastructure planning. Many public cloud providers are developing tools and programs to ease such migrations, including “bring your own IP” programs that make it easier to preserve sending reputation when making the move from the datacenter to the cloud.
On-Prem Email is Alive and Growing
The on-prem email ecosystem started with open source options such as Sendmail, Postfix, Qmail, and Exim. Commercial providers started appearing for inbound filtering that could be used for sending such as IronPort and ColdSpark, and sender solutions started to appear on the market such as the StrongMail MTA, Message Systems Ecelerity, and PowerMTA by Port25.
In recent years the options for on-prem senders has grown significantly, with several vendors entering the space, often with mature products that were prevalent in other markets but well suited to the high-volume on-prem sender. We plan to explore them here in more detail in 2021, but the list is impressive:
- SparkPost – An on-prem vendor that also provides cloud email relay services. Originally MessageSystems, creator of eCelerity (now Momentum), and by acquisition also the vendor for PowerMTA, formerly provided by Port25.
- MailerQ – A relatively new solution based on a unique AMQP architecture that allows for the deployment of a distributed MTA platform.
- SocketLabs – Creator of HurricaneMTA and one of the new hybrid vendors that provides both on-prem and cloud-based solutions.
- Halon – Originally focused on the receiver space, their MTA is geared toward policy management, making it a very flexible platform for senders.
- GreenArrow – One of the few vendors to provide both an MTA as well as a campaign management tool, Green Arrow started by creating highly optimized builds of Open Source tools to make them ready for commercial senders.
- EmailSuccess – An ESP that also provides and on-prem MTA, EmailSuccess is a European vendor that is making in-roads into the North American marketplace.
This growth in on-prem MTA vendors is creating a wealth of opportunities for on-prem email senders to identify the right solution for their needs without being limited to a small pool of options.
How to Succeed With On-Prem Email
A favourite line from the movie Ghostbusters sums up what it takes to succeed in on-prem email:
We had the tools, we had the talent!
Here at EmailNinjas, we focus on helping on-prem senders reach the inbox on time, every time. We do this by ensuring that our clients not only have access to the best tools available, but to the critical talent necessary to manage on-prem email sending at scale.
We are a collection of experienced email infrastructure engineers, deliverability specialists, content experts, and project managers who know on-prem and leverage over 100 years of collective experience to ensure senders are successful.
Whether you’re looking for remote management, product selection assistance, deliverability advice and remediation, or content development, we’ll work with you to make sure your on-prem email systems run smoothly, without the need for in-house expertise.
If you’re looking to take your on-prem email to the next level, contact us today!