I recently hired a new Marketing Manager, and she has been very busy during the past few weeks.  We have been revamping marketing materials, honing our positioning, getting more involved with the press (just published another release about Version 6.0 of our solution this morning), and performing more competitive analysis.  In the latter category, we have been reminded of some of the misleading information that is presented to marketers by email service providers with regard to deliverability.

You have undoubtedly seen these claims before: An ESP shares a benchmarking report from Pivotal Veracity or a similar deliverability guru that shows their customer's performance soundly dispatching its B-to-B or B-to-C counterparts.  In the seed-box test, the inbox percentage will be in the high 90's and the "bulk" and "missing" percentages will be negligible.  The conclusion that the marketer should supposedly derive is that the ESP is somehow superior to its competitors when it comes to deliverability.  "Sign up with us, and your messages will be delivered.  Sign up with those guys, and you'll spin your wheels by sending emails that miss their subscribers."  We could easily employ this tactic, as we have plenty of audits that highlight WhatCounts while displacing our competitors.  But, the truth is that the deliverability playing field is level, and these types of studies are misleading.

Sure, collateral sometimes makes numbers dance, but when numbers go from foot-tapping to circus contortion, it's important for marketers to be forewarned.  If a student gets an "A" on one exam one day, does that make him/her the most brilliant person at his/her school?  Furthermore, what if the exam he/she took was not even standardized with the exams that the other students received?

Deliverability audits show the performance of a particular piece of creative to a particular subscriber list from a particular IP address over a particular range of time.  They are the equivalent of a personalized exam for only one student at a school.  In order to standardize such an audit, the same exact creative would have to be sent to the same exact subscriber list from various email service providers a statistically significant number of times.  The last time I checked, there was no Scholastic Aptitude Test for deliverability.  It's a bunch of one-off tests that can range from 0% to 100% on any given day, depending on many factors (some of which are not within the email service provider's control).

So how can you measure one ESP's deliverability against another's?  There are three criteria that I can suggest:

• Take a look at their approach to ISP relationship management.  Are they whitelisting dedicated IP addresses for each customer, joining feedback loops, doing their part to help customers benefit from authentication programs, etc?  Also, how are they managing subscriptions, unsubscriptions, bounces, etc? 
• What are they doing to help customers employ best practices?  Deliverability is a team sport, and a customer who doesn't play fair can land their ESP in the penalty box.  So, verify to make sure they make it easy for you to score content, perform data hygiene, allow subscribers to manage their preferences, etc.  Also, make sure that they are investing time in proactively educating customers on best practices.  A team's performance is directly related to its level of preparation.
• What is the ESP doing to stay on top of industry trends?  Are they members of groups such as ISIPP or the ESPC?  Are they partnered with deliverability service providers like Pivotal Veracity, ReturnPath, or Habeas?  Do they offer third-party deliverability auditing to their customers to enter an objective party into the equation?

Of course, you could run a deliverability test amongst your top two ESP finalists before making your decision.  But, that would be like having two students take an exam on the first day of school...their results need some time to stabilize (for better or for worse).