Variations in email client displays

Email clients very rarely have their own HTML rendering engine (that's the part that reads your HTML and displays it to the user), this is most likely because building and maintaining an HTML rendering engine is actually an enormous undertaking that many email client manufacturers chose to outsource to an existing solution.

Before we discuss these rendering engines, here's a quick look at what an email client is doing when a user opens an HTML email.

The mail server

Each email client has to get the email somehow. In most cases, this step won't change your HTML, but there a few exceptions, we've detailed the different methods of collecting an email below.

POP & IMAP

POP and IMAP are protocols that an email client may use to talk to a mail server. The server will deliver the raw email, without adjusting its content, to any email client capable of speaking POP/IMAP. While these two protocols have significant differences, neither touch the HTML of the email, so as far as we're concerned they're the same. If you've ever collected your email from a webmail account onto, say, your mobile phone, chances are you used either POP or IMAP. These protocols are harmless in terms of rendering, they're also thought to be the most commonly used methods of retrieving email, so that's good news!

WebDav & DeltaSync

DeltaSync is a proprietary protocol used by Microsoft. Email clients such as Windows Live Mail Desktop will use DeltaSync (and previously used WebDav, an older alternative) to fetch email from web-based services such as Hotmail. For our purposes, it's safe to consider it identical to POP and IMAP in the sense that it doesn't adjust our HTML in any way.

Exchange

Exchange is Microsoft's email server software that manages (among other things) the sending and receiving of email. Unless particular filtering is used, Exchange will deliver the email to the email client without changing it. More good news!

Domino

Domino is IBM's email server software, similar to Exchange, specifically designed for use with Lotus Notes. Domino will indeed change an email's HTML to prepare it for viewing in Lotus Notes. These adjustments include removing any unsupported HTML elements, including OBJECT, IFRAME, FRAME(SET) and SCRIPT. Domino will also break any external links in an email, although the Lotus Notes client will repair these once the user selects the Show images link.

Preprocessing

Before an email's HTML can be handed over to a rendering engine, the client (or, in the case of Domino, the server) needs to pass the HTML through a preprocessor. A preprocessor's job sounds simple enough on the surface - remove anything that could be dangerous, introduce privacy concerns or cause the email client to behave unexpectedly (there are some wonderfully worrying examples of preprocessor mistakes in older email clients that we'll cover briefly later). Preprocessors can be broken into two categories.

Local

These are the preprocessors used by desktop and mobile email clients, such as Outlook or Mail on the iPhone. These preprocessors will very rarely cause headaches for an HTML email designer. They typically do most or all of the following:

• Remove javascript or any other client-side scripting.
• Remove object, embed and related tags. This prevents embedding ActiveX or Flash content within an email.
• Remove unrecognized tags. This is less common, but is somewhat sensible. The theory here is that when an email client is released, it doesn't retain control over the capabilities of the HTML rendering engine it's using. So if the email client left in a currently unknown tag, that was later introduced to the HTML spec and then support was added to the rendering engine, the email client could now be vulnerable to rendering content that was never supposed to be in an email. An interesting example would be Outlook 2003 that, when launched, had no knowledge of the VIDEO HTML element. But because Outlook 2003 borrows its HTML rendering engine from the version of Internet Explorer installed on the user's machine, Outlook 2003 would suddenly start supporting VIDEO as soon as IE does. Now, that might not sound like a bad thing, but imagine instead of VIDEO, we were discussing a brand new SCRIPT tag that supported a different kind of javsacript-like language. Suddenly that's considerably less desirable. So to protect against this disconnect between the email client and the rendering engine, preprocessors will strip out elements that they chose not to support or that they simply don't recognize.
• Break all external references (URLs) to prevent the rendering engine from loading external content without the user's permission. This is done to protect the user's right to privacy when opening an email.

Web-based

Web-based email clients (such as Outlook.com or Gmail) will certainly perform most the tasks listed above, but they also have a larger, more difficult job. They have to prepare the HTML to be converted into HTML that's safe to show within their own email client's HTML. This is a surprisingly difficult thing to do and most web-based email clients' preprocessors will err on the side of being overly restrictive and removing anything with even the slightest potential to affect the layout of their email client. We'll cover the various mistakes made by these preprocessors a little later, but you should expect this to be the second biggest cause of headaches when designing HTML emails - the first biggest cause is still to be discussed.

There are many web-based preprocessors, so we won't discuss them all, instead we'll use Hotmail as an example of how web-based email clients preprocess your HTML, and cover some of the mistakes they can make.

Outlook.com uses a preprocessor called MS SafeHTML. It's also used by Outlook Web Access, the web-based email client that ships with MS Exchange. SafeHTML takes a fairly typical approach to preprocessing that's important to understand as it can help to explain why it sometimes behaves unusually. To better explain this process, we've described how SafeHTML might handle an example HTML email. Let's start with our HTML:

 <html> <head> <style> body { border: 1px solid black; background-color: #999; width: 700px; height: 500px; } div#mydiv { border: 1px solid black; width: 500px; height: 300px; color: white; } .unused { background-color: black; } </style> <script type="text/javascript"> alert('boo'); </script> <body> <div id="mydiv" style="background-color: #555;"> Here is my <strong>email!</strong> </div> <object width="480" height="390"><param name="movie" value="http://www.youtube.com/v/6OBpiKE-qZ0?fs=1&hl=en_US"></param><param name="allowFullScreen" value="true"></param><param name="allowscriptaccess" value="always"></param><embed src="http://www.youtube.com/v/6OBpiKE-qZ0?fs=1&hl=en_US" type="application/x-shockwave-flash" allowscriptaccess="always" allowfullscreen="true" width="480" height="390"></embed></object> <iframe src="http://google.com/">No frames please!</iframe> <unknowntag>This isn't a known HTML element</unknowntag> </body> </html>

A trivial example, but enough to highlight how a preprocessor will tackle certain types of HTML. We can't be certain in exactly what order these things happen, so we've estimated based on the most sensible order.

First of all, SafeHTML needs to bring everything into the BODY. It doesn't process HEAD elements any different to BODY elements, so it makes sense to bring them all below the opening BODY tag before going any further. An interesting side effect of this is that if you've used anything in your HEAD that Hotmail doesn't later remove, it will render within your email's body. Next, it needs to handle blocklisted elements. These include SCRIPT, OBJECT, PARAM and EMBED. SafeHTML considers these dangerous and won't even keep their inner text (in this case, our Javascript). This is quite a brutal approach to preprocessing, which is only used for these special cases. After this stage, our HTML looks a little different:

<style> body { border: 1px solid black; background-color: #999; width: 700px; height: 500px; } div#mydiv { border: 1px solid black; width: 500px; height: 300px; color: white; } .unused { background-color: black; } </style><div id="mydiv" style="background-color: #555;"> Here is my <strong>email!</strong> </div> <iframe src="http://google.com/">No frames please!</iframe> <unknowntag>This isn't a known HTML element</unknowntag>

For less serious elements, SafeHTML (like most other preprocessors) reverts to slightly different behavior reserved for unknown elements. When SafeHTML encounters an element name that it doesn't have in its allow list of acceptable elements, it has to remove the element (just in case it's something potentially dangerous), but it will keep the inner text. In our example, both IFRAME and UNKNOWNTAG are removed, but the content of these elements is kept.

<style> body { border: 1px solid black; background-color: #999; width: 700px; height: 500px; } div#mydiv { border: 1px solid black; width: 500px; height: 300px; color: white; } .unused { background-color: black; } </style><div id="mydiv" style="background-color: #555;"> Here is my <strong>email!</strong> </div> No frames please! This isn't a known HTML element

The next problem to overcome is that some of those style rules will conflict with Outlook.com's own website, especially those targeting BODY. To fix this, SafeHTML first wraps the HTML in a div with class "ExternalClass" (there's a little extra to this div, but we can ignore this for now):

<div id="mpf0_MsgContainer" class="SandboxScopeClass ExternalClass"> <style> body { border: 1px solid black; background-color: #999; width: 700px; height: 500px; } div#mydiv { border: 1px solid black; width: 500px; height: 300px; color: white; } .unused { background-color: black; } </style><div id="mydiv" style="background-color: #555;"> Here is my <strong>email!</strong> </div> No frames please! This isn't a known HTML element </div>

SafeHTML then needs to adjust the CSS rules. It starts by prepending the class qualifier ".ExternalClass" to every rule (it also removes any mention of "body" at this point):

<div id="mpf0_MsgContainer" class="SandboxScopeClass ExternalClass"> <style> .ExternalClass { border: 1px solid black; background-color: #999; width: 700px; height: 500px; } .ExternalClass div#mydiv { border: 1px solid black; width: 500px; height: 300px; color: white; } .ExternalClass .unused { background-color: black; } </style><div id="mydiv" style="background-color: #555;"> Here is my <strong>email!</strong> </div> No frames please! This isn't a known HTML element </div>

Next, it prepends "exc" to every id and class used within the HTML. Unfortunately, it does this in a fairly crude way, literally prepending "exc" to the ids and classes, then proceeds to update each CSS rule in exactly the same way:

<div id="mpf0_MsgContainer" class="SandboxScopeClass ExternalClass"> <style> .ExternalClass { border: 1px solid black; background-color: #999; width: 700px; height: 500px; } .ExternalClass excdiv#mydiv { border: 1px solid black; width: 500px; height: 300px; color: white; } .ExternalClass .excunused { background-color: black; } </style><div id="excmydiv" style="background-color: #555;"> Here is my <strong>email!</strong> </div> No frames please! This isn't a known HTML element </div>

Did you notice the problem it just caused? It added "exc" to the CSS rule "div#mydiv" making it "excdiv#mydiv". Obviously this won't work as there's no such element as excdiv, as a result, all of the rules that were being applied to mydiv will no longer work in Outlook.com or Outlook Web Access. Just one example of how preprocessing can cause some confusing side effects!

At this point, the email is ready to be displayed within Outlook.com safely. Outlook.com drops the adjusted HTML straight into its interface, confident it won't disrupt the Outlook.com UI.

While this is just one example of a preprocessor, and other preprocessors have their own limitations, it does reflect a fairly typical approach to preprocessing for web-based email clients.

Rendering engines

Once the email client has got the email and has preprocessed it appropriately, it's now ready to hand the HTML over to the rendering engine to be properly processed and shown to the user. Rendering engines come in a few different flavors:

Trident (Internet Explorer)

Microsoft's Internet Explorer uses the rendering engine known as Trident. It was built for Internet Explorer and is used by hundreds of different applications to render HTML. It includes support for disabling ActiveX and Javascript without the need for the preprocessor to remove these HTML elements, which certainly takes the burden off of the email client to ensure these potentially dangerous features can't be used.

One downside of the Trident rendering engine is that until very recently (Internet Explorer 7) it was not possible to bundle Trident with your application, instead manufacturers had to ask Windows to borrow its default installed version. This has led to some unfortunate side effects. One example that comes to mind is the McAfee Anti-Virus software. McAfee's user interface is written in HTML, which is then passed to Trident which renders it. When Microsoft released Internet Explorer 7, McAfee's UI HTML was written specifically for IE6 and any user who installed IE7 would see their copy of Trident replaced with the latest version. Unfortunately, this latest version wasn't compatible with McAfee's HTML and the application became unusable until an update was hastily released.

This same issue is apparent in Outlook 2003. When Outlook 2003 was launched, almost all of its users were running IE6 and the version of Trident that went with it. Since then, Microsoft have released 3 new versions of IE, Trident looks almost nothing like it did back when IE6 was around.

You might think this means that when testing your HTML email in Outlook 2003, you need to test 4 different copies (Outlook 2003 installed on a machine with IE6, another with IE7 etc), but actually it's a bit simpler than that. For every designer's sanity, Microsoft introduced 'IE7 compatibility view', then forced Trident to revert to this mode whenever embedded in an application other than Internet Explorer. This means that, realistically, you only need to test in Outlook 2003 with IE7. Even if IE8 or IE9 is installed, Outlook will continue to use the IE7 version of Trident.

Outlook Express and Windows Live Mail Desktop also use Trident in an identical fashion to Outlook 2003.

It's important to note that Outlook 2007, 2010, and 2013 do not use Trident at all, we'll cover that in more detail later.

Lotus Notes also takes advantage of the Trident rendering engine, which sounds fantastic on the face of it - you only have to design for Outlook 2003 and your email will work in Lotus Notes too! Regrettably, this isn't the case; There are two flavors of Lotus Notes. The first uses the Lotus Notes built-in POP or IMAP support to connect to a normal email server. This is how most email clients work. The second uses a Domino server to retrieve email.

Lotus Notes will render your email differently depending on the follow considerations:

NotesRichText

The NotesRichText rendering engine is a heavily limited HTML rendering engine. It works by converting your email's HTML into a proprietary rich text format and displaying supported elements in a special rich text viewer. It's akin to Microsoft's Word HTML rendering engine, but unfortunately supports an even smaller subset of HTML elements, attributes and CSS rules. Incredibly, it does include support for some oft-disabled in HTML email elements, including IFRAME, although it's strongly discouraged to attempt to use these elements as their support is weak at best and is likely to trigger warnings in other email clients and spam filters. The best approach to working with the NotesRichText client is to inline your CSS into style attributes where ever possible, to 'reset' layout style rules, such as padding and margin and to use table-based layouts.

No other email clients use NotesRichText and IBM have stopped developing this particular rendering engine.

Gecko

Mozilla's Gecko rendering engine is the engine used by Firefox. It's a very modern, well maintained, actively developed rendering engine with top-notch support for all modern web standards. Unlike Trident, Gecko allows you to bundle a fixed version with your application, and the email client Thunderbird does just this. This does mean that earlier versions of Thunderbird will be using an older version of Gecko, but with Thunderbird's upgrade uptake being very high, this is unlikely to be an issue for you. As a general rule of thumb, if it works in Firefox, and Thunderbird's preprocessor doesn't strip it out, it'll work in Thunderbird too.

Tasman

Microsoft's Tasman rendering engine was used by Entourage 2004 (the version of Outlook for Mac OS X). It was originally designed for Internet Explorer 5 for Mac and was actually considered to be quite ahead of its time. Embedding it within Entourage 2004, however, was seen as a significant limitation of the Entourage client and when Microsoft launched Entourage 2008, they chose to drop Tasman in favor of WebKit.

WebKit

On par with Gecko, WebKit is unlikely to give you much trouble when designing HTML emails. Email clients that use WebKit include Apple Mail, Entourage 2008 and Outlook for Mac 2011.

There's also a mobile version of WebKit, with similarly superior rendering capabilities that's used by Mail on the iPhone and iPad, Symbian, very recent versions of BlackBerry's email client and many Android email clients.

Microsoft Word 

Historically, Microsoft Outlook used Internet Explorer (Trident) as its rendering engine, and Microsoft Word for composing email. Word was chosen for composing emails because of Word's focus on authoring rich documents and its ability to embed MS Office content, such as spreadsheets, charts and other media. This created a problem for Outlook users, they could create great-looking HTML emails in Outlook, but when emailing another Outlook user the email would render completely differently. Even when opening the email in their sent items folder, it would look different to the email they had just sent! This understandably caused some frustration among larger organisations who had unified installations of Outlook across the company. Pressure for Microsoft to fix the issue increased. As the embeddable content from Office applications grew increasingly complex in later Office versions, the burden to add support for these to Trident became a little too much.

The first attempt at fixing the issue was found in Outlook 2003, Microsoft added a couple of additional, advanced options to allow users to opt-out of using Word for authoring emails, or to opt-in to using Word to render HTML emails. However, since most users never saw these options, let alone understood the relevance of them (they also had a confusing dependency on Word 2003 being installed, even though Word 2003 is not required to write emails in Word, which was the default. In fact, Word 2003 was required just to opt-out of composing in Word!), the problem remained as much of an issue as it ever was.

By 2006, and Microsoft's release of Outlook 2007 Beta, Microsoft had dropped Trident in favor of using Word for both rendering and composing HTML emails. While this fixed their issue of rendering discrepancies between the same versions of Outlook, it created a much larger issue - the Word rendering engine is very poor at rendering HTML that wasn't originally created in Word. This was terrible news for HTML email designers everywhere and the design community was understandably upset. With Word 2007/10's market share increasing each day as more Outlook 2003 users upgrade, Outlook 2007 and above is likely to cause headaches for many years to come.

Microsoft's own MSDN documentation site lists some of the many limitations of the Word rendering engine.